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This  publication  is  primarily  a  working  paper. 

It  is  published  solely  to  document  work  performed. 


PREFACE 


The  central  goal  of  this  project  is  to  demonstrate  the  existence  and 
utility  of  a  systematic  structure  of  flight-related  concepts  of  Air  Force 
fighter  pilots.  This  paper  reviews  work  on  defining  and  measuring 
conceptual  structures  and  on  assessing  the  reliability,  validity,  and 
utility  of  particular  structural  descriptions.  This  work  is  part  of  6.1 
basic  research  that  is  intended  to  advance  the  understanding  of  basic 
cognitive  dimensions  as  represented  in  flying  behavior. 

Several  individuals,  in  addition  to  the  authors,  have  made  important 
contributions  to  this  work.  At  New  Mexico  State  University,  Karen  Preuss 
has  assisted  with  the  project  in  countless  ways,  and  Don  Dearholt  provided 
valuable  advice  on  properties  of  networks. 

The  cooperation  of  many  pilots  and  other  Air  Force  personnel  is  grate¬ 
fully  acknowledged. 
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CONCEPTUAL  STRUCTURES  IN  FIGHTER  PILOTS 


INTRODUCTION 

The  central  goal  of  this  project  is  to  demonstrate  the  existence  and 
utility  of  a  systematic  structure  of  flight-related  concepts  in  the  memory 
systems  of  Air  Force  fighter  pilots.  This  paper  constitutes  the  second 
annual  report  on  the  project.  First,  the  major  findings  from  the  first  year 
of  the  project  (Schvaneveldt,  Goldsmith,  Durso,  Maxwell,  Acosta,  &  Tucker, 
1982'  will  be  summarized,  and  then  an  overview  of  the  work  accomplished  this 
year  will  be  presented.  Background  information  relevant  to  each  aspect  of 
the  overall  project  is  presented  in  the  appropriate  sections. 

The  first  year  of  the  project  was  devoted  to  the  task  of  defining  and 
measuring  conceptual  structures  and  to  assessing  the  reliability,  validity, 
and  utility  of  particular  structural  descriptions.  The  assessment  of 
conceptual  structures  required  a  data  base  that  could  be  used  in  our  efforts 
to  develop  reliable  and  valid  techniques  for  describing  the  psychological 
organization  of  flight-related  concepts. 

The  stimulus  material  for  the  project  was  provided  by  30  concepts 
relating  to  split-plane  maneuvers  and  30  concepts  relating  to  the  low  angle 
strafe  maneuver.  These  concepts  were  selected  with  the  assistance  of  senior 
instructor  pilots  (IPs)  at  Holloman  AFB.  Ratings  of  the  psychological 
similarity  of  the  concepts  were  obtained  from  four  groups  of  officers:  Air 
National  Guard  Pilots  (GPs) ,  Fighter  Lead-In  Instructor  Pilots  (IPs), 
recent  Undergraduate  Pilot  Training  graduates  (UPs),  and  Instructor  Weapons 
Systems  Officers  (IWs).  Cognitive  structures  were  defined  by  two  analytic 
procedures:  multidimensional  scaling  (MDS)  and  general  weighted  networks 
(GWNs).  The  GWN  algorithm  was  developed  as  part  of  the  project.  The  MDS 
solutions  represent  concepts  as  points  in  multidimensional  space.  The  GWN 
networks  represent  concepts  as  nodes  and  relations  as  links  connecting  the 
nodes . 

Three  dimensions  in  the  MDS  solutions  were  identified:  the  order  in 
which  concepts  are  considered  in  an  air-to-air  encounter,  a  contrast  between 
lead  and  lag  pursuit,  and  the  temporal  order  of  events  in  a  training 
sequence.  A  pattern  recognition  algorithm  was  applied  to  the  similarity 
ratings  and  MDS  solutions.  This  analysis  showed  that  group  membership  can 
be  predicted  from  a  person's  conceptual  structure.  Reliable  predictions 
were  made  when  a  subset  of  the  members  of  each  group  was  used  to  define  the 
pattern  classifier,  and  the  remaining  individuals  were  classified. 
Classification  was  significantly  better  with  MDS  results  than  with  raw 
similarity  ratings. 
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The  networks  produced  by  the  GWN  algorithm  were  analyzed  using  various 
constructs  from  graph  theory.  Minimum  cycles  in  the  IP  network  corresponded 
to  clusters  of  interrelated  concepts.  Nine  substructures  were  defined  by 
the  agreement  in  the  IP  and  GP  networks.  Finally,  the  UP  network  was 
compared  to  the  IP  and  GP  networks,  resulting  in  the  identification  of 
specific  points  of  agreement  and  disagreement  in  the  conceptual  organization 
of  novice  and  expert  pilots. 

On  the  basis  of  the  initial  year  of  the  project,  it  was  concluded  that 
pilots  do  have  measurable  cognitive  structures  for  organizing  flight- 
related  information.  These  structures  are  measurably  different  for 
individuals  with  differing  flight  experience.  The  IPs  exhibit  more 
efficient  and  economical  structures  than  do  UPs.  The  techniques  employed  in 
the  research  produce  descriptions  of  conceptual  structure  that  may  have 
application  in  the  training  program  of  fighter  pilots  and  in  assessing 
individual  differences  in  the  development  of  appropriate  conceptual 
structures.  The  specific  differences  between  novice  and  expert  pilots 
3hould  provide  assistance  in  planning  training  programs  for  fighter  pilots. 

In  the  second  year  of  the  project,  investigation  continued  into  the 
properties  of  multidimensional  scaling  and  network  representations  of 
conceptual  structure.  These  efforts  have  included  further  development  of 
the  GWN  algorithm.  A  procedure  to  obtain  metric  information  from  a  network 
has  been  developed,  and  several  options  for  defining  path  length  in  a 
network  have  been  added  and  related  to  assumptions  about  properties  of  data 
used  in  constructing  the  network.  The  ability  to  discriminate  between 
individuals  in  different  groups,  given  the  network  structures  for 
individuals,  was  also  examined.  The  discrimination  possible  with  networks 
was  compared  to  the  discrimination  obtained  with  MDS  and  the  original  rating 
data.  The  relations  represented  by  the  links  in  the  IP  network  have  been 
identi fied . 

A  new  procedure  for  determining  the  optimum  number  of  dimensions  to  use 
in  MDS  was  developed  and  used  to  examine  the  data  collected  last  year. 
AI30,  a  multidimensional  representation  of  individuals  was  derived  that 
located  each  individual  from  each  group  as  a  point  in  two-dimensional 
space.  This  procedure  can  be  used  to  determine  how  well  individuals  from 
different  groups  are  aligned  with  other  members  of  their  group. 

Two  experiments,  comparing  the  structures  derived  with  MDS  and  GWN,  were 
conducted.  One  of  them  suggested  that  GWN  better  captures  the  relationships 
between  pairs  of  concepts.  This  result  would  be  expected  on  the  basis  of 
the  differences  in  the  two  scaling  procedures.  The  other  experiment 
revealed  some  methodological  problems  that  accompany  the  use  of  "priming" 
techniques  in  investigating  the  structure  of  a  small  set  of  concepts. 
Finally,  a  new  analysis  of  the  individual  concepts  further  defines  which 
concepts  are  critical  to  distinguishing  between  novices  and  experts. 


NETWORKS 


A  OWN  is  a  configuration  where  concepts  are  depicted  by  nodes  and 
relations  are  depicted  by  links  connecting  the  nodes.  The  links  are  assigned 
a  value  or  weight  that  reflects  the  strength  of  the  relationship  between  the 
nodes.  The  value  indicates  the  distance  from  one  node  to  another  along  that 
link  and  is  an  index  of  the  psychological  proximity  or  relatedness  of  the 
concepts  represented  by  the  nodes. 

During  the  previous  reporting  period,  an  algorithm  was  developed  and 
implemented  which  constructed  GWNs  from  empirical  similarity  judgments 
(Schvaneveldt,  et  al . ,  1982).  The  algorithm,  GWN,  was  applied  to  judgments 
taken  from  UPs,  IPs,  and  GPs  and  produced  networks  for  each  of  these  groups. 
GWN  supplies  several  important  pieces  of  information.  First,  like  the  other 
scaling  techniques,  GWN  summarizes  the  data  to  a  considerable  extent.  The 
results  of  GWN  are  often  considerably  more  inter  pretable  than  other  scaling 
techniques,  and  in  addition,  GWN  offers  options  (e.g.,  minimally  connected 
networks)  that  allow  for  even  further  data  reduction.  Second,  GWN  captures 
the  local  relationships  among  the  concepts.  Unlike  MDS,  GWN  focuses  on  the 
item-to-item  relations  in  constructing  the  network.  This  had  both  adva  >ges 
(e.g.,  keeping  close  to  the  data)  and  disadvantages  (e.g.,  li-  was 
dependent  on  a  small  portion  of  the  data). 

During  the  current  reporting  period,  a  metric  that  was  developed  ,)  GWN 

uses  more  information  from  the  network  to  derive  weights  for  the  '  s 

between  concepts.  The  metric  information  may  be  more  useful  th  .ie 

empirical  judgments  for  the  purpose  of  identifying  the  experience  of 

pilots.  It  will  also  be  interesting  to  compare  the  network  metric  to  the 
MDS  metric.  The  appropriate  metric  for  a  network  depends  on  the  scale 
properties  of  the  original  similarity  judgments.  Most  scaling  procedures 
assume  that  the  original  judgments  are  ordinal  (i.e.,  the  numbers  contain 
information  about  the  ranks  of  the  concepts).  The  GWN  allows  the  user  to 
specify  different  definitions  of  minimun  path  length  depending  on  the  scale 
properties  of  the  data.  Finally,  GWN  produces  networks  that  can  be 
decomposed  into  substructures  (e.g.,  cycles,  trees,  assemblies).  The 
substructures  are  useful  in  gaining  a  further  understanding  of  the  network. 
These  substructures  were  used  in  the  last  reporting  period  to  determine 
which  concepts  held  by  the  UPs  were  not  well  defined  relative  to  the 
ex perts. 


Analytic.  Work  on  General  Weighted  Networks 

There  have  been  two  central  thrusts  in  the  development  of  network 
analyses  of  conceptual  structures.  First,  the  definition  of  path  length  in 
a  network  has  been  generalized  to  take  considerations  of  levels  of 
measurement  into  account.  Second,  a  metric  that  was  developed  for  the 
network  maximizes  the  fit  of  the  network  model  to  the  empirical  proximities. 
Each  of  these  efforts  is  presented  in  this  section. 
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Path  Length  and  Levels  of  Measurement 


The  definition  of  the  length  of  a  path  in  a  network  has  been  one  of  the 
central  problems  in  developing  methods  for  deriving  networks  from 
psychological  proximity  data.  Recent  efforts  on  this  problem  have  attempted 
to  relate  the  definition  of  path  length  to  the  information  contained  in  the 
data . 


Stevens  (1951)  proposed  a  measurement  scales  taxonomy  that  has  been 
widely  used  over  the  years.  His  taxonomy  includes  nominal  scales  that 
simply  identify  the  category  appropriate  for  each  element,  ordinal  scales 
that  have  the  property  of  ordering  the  elements  without  providing  infor¬ 
mation  about  the  distance  between  elements,  interval  scales  that  have  both 
order  and  distances  information,  and  ratio  scales  that  have  a  meaningful 
zero  point  in  addition  to  the  order  and  interval  information.  One  useful 
perspective  on  levels  of  measurement  is  found  in  the  class  of  transform¬ 
ations  that  can  be  applied  to  the  data  without  losing  information  of  various 
sorts.  Nominal  scales  permit  any-one-to— one  transformation  without  loss  of 
nominal  information.  Ordinal  scales  permit  any  (positive)  monotonic 
transformation  without  loss  of  ordinal  information.  Interval  scales  permit 
only  (positive)  linear  transformations ,  and  ratio  scales  permit  only 
multiplication  by  a  (positive)  constant. 

For.  purposes  of  this  study,  the  ordinal  and  interval  scales  hold  the 
most  interest  since  it  is  assumed  that  data  of  interest  would  convey  at  least 
the  appropriate  order  of  elements.  Whether  the  data  also  have  interval 
properties  is  open  to  question.  The  current  version  of  the  algorithm  has 
three  definitions  of  path  length  that  require  different  assumptions  about 
the  data.  The  algorithm  also  yields  a  minimal  connected  network  (MCN) 
that  requires  only  ordinal  properties  in  the  data.  Only  the  elaborated 
networks  are  affected  by  the  presence  or  absence  of  interval  information  in 
the  data. 

The  additive  definition  of  path  length  involves  simnly  adding  the  values 
of  the  links  in  the  path  to  arrive  at  the  length  of  the  path.  The  operation 
of  addition  and  the  comparison  of  a  sum  to  another  data  point  require 

interval  data.  In  practice,  the  additive  rule  produces  networks  with  many 
links,  which  i3  not  necessarily  bad,  but  it  represents  the  data  as  concisely 
as  do  the  other  rules.  Networks  resulting  from  application  of  the  additive 
rule  are  invariant  over  linear  transformations  of  the  data.  Thus,  if  only 

ordinal  information  is  provided  by  the  data,  the  additive  rule  is  not 

appropriate . 

The  maximum  definition  of  path  length  involves  using  the  value  of  the 

maximum  link  in  a  path  as  the  length  of  the  path.  This  rule  uses  only 
ordinal  properties  of  the  data,  and  the  network  resulting  from  applications 
of  the  maximum  rule  is  invariant  with  any  ( pos* tive)  monotonic  transform¬ 
ation  of  the  data.  The  maximum  rule  also  produces  the  minimal  unique 
network.  The  MCN  is  not  necessarily  unique  when  there  are  ties  in  the 
data.  Since  these  ties  are  included  using  the  maximum  rule,  the  resulting 
network  is  unique. 


Another  definition  of  path  length  that  requires  only  ordinal  properties 
of  the  data  is  the  use  of  ranks  rather  than  data  values.  When  these  ranks 
are  used  with  the  additive  rule,  the  resulting  network  is  invariant  with 
(positive)  monotonic  transformations  of  the  data.  Some  properties  of  these 
"rank"  networks  are  currently  being  investigated. 

Metric  Procedure  for  Networks 

Once  the  structure  of  the  network  is  determined  by  the  GWN  algorithm, 
the  link  weights  can  be  adjusted  by  a  linear  regression  procedure.  This 
adjustment  produces  the  best  fit  between  the  network  and  the  original  data 
by  regressing  the  links  in  the  network  on  the  original  data.  Each  pair  of 
items  from  the  distance  matrix  has  an  associated  empirical  distance  as  well 
as  a  minimum  path  in  the  network  that  is  defined  by  the  GWN  algorithm.  The 
metric  procedure  essentially  allows  the  value  of  each  link  to  be  determined 
by  its  role  in  each  minimum  path  of  which  it  is  a  member.  The  metric 
procedure  uses  an  additive  definition  of  path  length  where  the  length  of  a 
path  is  the  sum  of  the  weights  for  the  links  constituting  the  path. 

The  metric  weights  are  produced  by  a  regression  analysis  that  treats 
each  pair  of  terms  as  a  ca3e,  the  links  as  independent  variables,  and  the 
empirical  distances  as  the  dependent  variable.  The  values  of  the  independent 
variables  (links)  are  1  for  those  links  in  the  minimum  path  for  each  pair  of 
terms  and  0  for  the  links  not  in  the  minimun  path  for  the  pair.  The  metri- 
cized  weights  are  given  by  the  beta  values  (the  regression  coefficients) 
that  result  from  the  regression  analysis. 

Networks  represent  useful  structures  for  analyzing  the  conceptual 
organizations  in  various  domains.  An  attempt  is  being  made  to  place  the 
network  algorithms  on  a  firm  footing  to  support  the  use  of  them  in  this 
project  and  to  provide  an  analytical  tool  that  should  prove  to  be  generally 
useful.  During  the  past  year,  theoretical  efforts  have  concentrated  on 
developing  the  network  algorithm.  Now  comparison  can  be  made  between  the 
performance  of  networks  and  MDS  using  metric  information  from  each  method. 
The  following  study  reports  some  initial  work  on  evaluating  the  network 
without  using  metric  information.  Studies  using  the  metric  are  currently 
under way. 

Classification  of  Individuals  Using  Networks 

Similarity  ratings  between  pairs  of  concepts  related  to  a  particular 
knowledge  domain  and  MDSs  of  these  ratings  both  are  ways  of  describing 
conceptual  structures.  These  conceptual  structures  provide  relational  and 
organizational  information  about  the  concepts  within  a  domain  of  knowledge. 
Networks  also  provide  this  type  of  information.  The  comparison  of 
structures  across  groups  of  individuals  allows  for  the  determination  of 
qualitative  differences  between  groups.  This  qualitative  comparison  is 
important  for  revealing  how  different  groups  view  a  particular  set  of 
concepts. 


The  purpose  of  this  phase  of  the  project  is  to  determine  the  quanti¬ 
tative  differences  between  the  representations  of  concepts  in  the  networks 
of  individuals  as  well  as  groups.  The  previous  accomplishment  of  developing 
pattern  recognition  programs  for  determining  quantitative  differences  has 
allowed  for  groups  of  subjects  to  be  classified  based  on  similarity  ratings 
and  distances  in  MDS.  The  success  of  pattern  recognition  techniques  in 
classifying  individuals  into  groups  pro...Jed  validation  for  the  data  and  the 
derived  MDS  representations.  Using  the  pattern  classification  technique 
with  network  representations  provides  a  test  of  the  validity  of  networks  for 
representing  the  structure  of  flight-related  concepts. 

Pattern  recognition  analysis  provides  information  about  the  degree  to 
which  individuals  are  associated  with  each  group.  The  analysis  reveals  how 
close  an  individual  is  to  the  prototype  of  the  individual's  group.  AI30,  the 
distance  from  an  individual  to  the  decision  surface  separating  two  groups 
reveals  how  close  that  person  is  to  being  classified  as  a  member  of  another 
group  of  individuals.  Both  indices  provide  information  about  the  strength 
of  the  association  of  each  individual  with  various  groups. 

Method 

A  pattern  was  formed  for  each  individual  tested  by  taking  the  presence 
or  absence  of  links  in  the  network  for  each  pair  of  concepts.  All  the 
concepts  have  potential  links  to  all  other  concepts.  The  network  solution 
for  each  subject  yields  links  between  concepts  which  were  represented  as 
being  present  or  absent  in  the  network  by  a  1  or  a  0,  respectively.  An  MDS 

solution  yields  a  metric  formed  by  taking  the  distance  between  each  pair  of 

concepts  in  a  multidimensional  space.  Thus,  an  MDS  pattern  may  be  created 

by  viewing  the  attributes  of  the  pattern  as  values  of  the  metric.  This 

allows  the  pattern  for  the  MDS  solution  to  preserve  the  structural  proper¬ 
ties  inherent  in  the  MDS  solution.  While  the  pattern  for  the  network 
solution  does  not  contain  a  metric  for  the  length  of  a  link,  the  attributes 
of  the  pattern  do  retain  some  of  the  structural  properties  of  the  network 
through  the  presence  or  absence  of  links. 

A  third  type  of  pattern  may  be  generated  by  considering  the  similarity 
rating  for  each  pair  of  concepts  as  a  feature  of  a  pattern.  This  pattern, 
containing  the  individual  similarity  ratings,  lacks  the  structural 
properties  imposed  by  the  network  and  MDS  scaling  techniques.  However,  the 
pattern  for  similarity  ratings  would  possess  more  scaling  information  about 
the  degree  of  relatedness  of  any  two  concepts  than  would  the  network 
pattern.  This  sets  up  an  interesting  comparison  to  determine  if  there  are 
beneficial  aspects  in  having  a  structural  representation  of  knowledge  over  a 
non-structural  one,  even  when  the  latter  contains  scaling  information  and 
the  former  does  not.  All  three  methods  resulted  in  patterns  with  435 
features  corresponding  to  all  the  possible  pairs  of  30  concepts. 
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The  first  analysis  consisted  of  applying  a  minimum-distanc e  classifier 
to  all  pairs  of  groups  of  pilots  for  both  the  split-plane  and  low  angle 
strafe  maneuvers.  Prototype  points  for  all  groups  and  decision  surfaces  for 
separating  all  pairs  of  groups  were  generated.  In  each  application  of  the 
m  in  i -nun-distance  classifier,  all  members  of  the  two  groups  were  used.  This 
provided  information  about  both  group  and  individual  differences.  The 
distances  from  each  individual  to  a  decision  surface  and  from  each 
individual  to  the  group  prototypes  were  computed  along  with  the  distances 
between  group  prototypes. 

The  second  analysis  involved  computing  a  decision  surface  that  separated 
a  training  set  consisting  of  a  limited  number  of  members  from  two  groups  and 
then  applying  the  decision  surface  to  the  remaining  members  of  the  groups. 
Decision  surfaces  were  computed  with  a  training  algorithm  if  a  minimum- 
distance  classifier  did  not  separate  the  training  sets.  Weight  vectors  were 
initialized  to  the  weights  produced  by  a  minimun-distance  classification  of 
the  individuals  in  the  limited  training  set.  This  minimized  the  number  of 
iterations  needed  to  produce  a  solution  when  the  minimum  distance  classifier 
failed  and  also  Kept  the  final  weight  vector  as  close  to  the  minimum- 
distance  decision  surface  as  possible.  The  first  analysis  showed  that  the 
classes  clustered  tightly,  indicating  that  when  the  minimum-distance  weights 
failed  to  separate  the  classes,  a  solution  close  to  these  weights  was 
likely.  A  small  correction  increment  (c  =  .01)  was  also  used  to  produce 
minimal  change  from  the  minimum-distance  weights. 

For  each  pair  of  pilot  groups,  a  training  set  of  a  particular  size  was 
randomly  chosen,  and  a  decision  surface  was  computed  to  separate  the  members 
of  the  training  set  into  their  respective  classes.  In  the  case  where  a 
minimum-distance  classifier  correctly  separated  the  members  of  the  training 
sets,  the  resulting  discriminant  function  was  then  applied  to  the  remaining 
members  of  the  two  groups.  If  no  solution  was  found  with  the  minimum- 
distance  classifier,  the  training  algorithm  was  applied  to  the  subset  of 
selected  group  members  to  generate  a  decision  surface.  This  discriminant 
function  was  then  applied  to  the  remaining  members.  This  procedure  was 
repeated  100  times  for  each  training  set  size.  The  training  sets  consisted 
of  equal  numbers  of  individuals  from  each  group.  The  size  of  the  training 
sets  was  increased  until  the  size  of  the  smaller  group  was  reached. 

Results  and  Discussion 

The  results  of  the  pattern  recognition  analyses  performed  indicate  that 
it  is  possible  to  discriminate  classes  of  flying  personnel  based  on  their 
network  representations  of  critical  flight  information  for  both  split-plane 
and  low  angle  strafe  maneuvers.  Also  significant  was  the  result  that 
patterns  represented  by  the  networks  produced  better  group  separation  than 
did  patterns  based  upon  similarity  ratings. 

Table  1  shows  that  a  minimun-distance  classifier  applied  to  each  pair  of 
groups  resulted  in  well  separated  groups  with  no  erroneous  classifications 
for  the  network  patterns.  The  same  results  occurred  for  the  patterns 
representing  distances  in  an  MDS  solution,  whereas  the  decision  surface 
generated  for  separating  groups  on  the  basis  of  similarity  ratings  misclass- 
ified  some  individuals.  Even  with  the  simplicity  of  the  decision  surface 


-7- 


generated  by  the  minimum-distance  classifier,  the  results 
distinct  classes,  especially  in  the  case  of  IPs  and  UPs  where 
classified  as  UPs  and  no  UPs  were  classified  as  IPs  for  either 
any  of  the  three  pattern  types. 


indicate  very 
no  IPs  were 
maneuver  with 


Table  1 


Classifications  Based  on  Group  Separation 
with  a  Minimum-Distance  Classifier 


Split-Plane  Maneuvers 

Ratings  MDS  Distances  Networks 


IPs  classified  as 
GPs  classified  as 
IWs  classified  as 
UPs  classified  as 


IP  GP  IW  UP 

3  2  2  0 

0  9  0  0 

0  0  4  0 

0  4  1  12 


IP  GP  IW  UP 

7  0  0  0 

0  9  0  0 

0  0  4  0 

0  0  0  17 


IP  GP  IW  UP 

7  0  0  0 

0  9  0  0 

0  0  4  0 

0  0  0  17 


IPs  classified  as 
IWs  classified  as 
UPs  classified  a3 


Low  Angle 

Strafe 

Ratings 

MDS 

Distances 

Networks 

:p  IW  UP 

it 

IW 

UP 

IP 

IW  Uj 

6  0  0 

6 

0 

0 

6 

0  i 

0  6  1 

0 

7 

0 

0 

7  i 

0  3  13 

0 

0 

16 

0 

0  1i 

It  s  important  to  note  that  there  was  perfect  separation  of  groups 
using  both  the  patterns  from  MDS  and  the  network  solutions.  The  structural 
information  supplied  by  both  these  patterns  appears  to  maximize  the  class 
differences.  This  information  provides  support  to  the  claim  that  both  the 
MDS  and  network  solutions  extract  important  structural  information  from 
similarity  ratings. 

Similarity  between  groups  may  be  measured  by  the  distance  between  group 
prototypes,  as  shorter  distances  would  suggest  greater  similarity  between 
the  conceptual  structures  of  the  groups.  Table  2  contains  the  distances  for 
all  pairs  of  groups  and  a  ranking  of  these  distances.  The  relatively 
smaller  distances  between  prototypes  for  the  networks  is  due  to  the  fact 
that  they  are  determined  from  patterns  of  ones  and  zeros,  whereas  distances 
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between  group  prototypes  for  ratings  and  MDS  distances  are  each  on  a 
different  scale.  Direct  comparison  across  the  three  pattern  types  of  the 
distance  between  prototypes  would,  therefore,  not  be  very  useful.  The 
rankings  of  the  distances  between  prototypes,  however,  does  provide  infor¬ 
mation  about  the  similarity  of  group  conceptual  structures.  As  it  turns 
out,  the  distances  between  prototypes  for  the  MDS  distances  and  networks  are 
assumed  to  be  most  valid,  because  the  only  misclassifications  occurred  for 
patterns  using  the  ratings  information.  Table  2  shows  that  the  most  similar 
groups  are  IPs  and  GPs  for  the  split-plane  maneuvers,  and  IPs  and  IWs  for 
the  low  angle  strafe.  Consistent  with  the  finding  that  no  misclassifications 
occurred  between  IPs  and  UPs,  it  can  be  seen  that  these  two  groups  form 
dissimilar  classes. 


Table  2  -  Distances  Between  Group  Prototypes 
Split-Plane  Maneuvers 


Ratings 

Rank 

Distances  in  MDS 

Rank 

Networks 

Rank 

IP-GP 

33.36 

2 

81.30 

1 

4.24 

1 

IP-IW 

38.02 

4 

82.96 

2 

5.13 

4 

GP-UP 

37.90 

3 

91.31 

3 

4.35 

2 

GP-IW 

30.72 

1 

92.56 

4 

5.51 

5 

IP-UP 

39.14 

5 

106.29 

5 

4.98 

3 

IW-UP 

49.41 

6 

111.96 

6 

5.72 

6 

Ratings 

Rank 

Low  Angle  Strafe 

Distances  in  MDS 

Rank 

Networks 

Rank 

IP-IW 

25.95 

1 

76.32 

1 

4.41 

1 

IW-UP 

36.10 

2 

95.87 

2 

4.94 

2 

IP-UP 

38.97 

3 

120.83 

3 

5.15 

3 
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In  addition  to  providing  information  about  differences  between  groups, 
pattern  classification  analyses  also  make  available  information  on  indiv¬ 
idual  members  of  the  groups  that  have  been  classified.  Each  individual  is 
represented  as  a  point  in  the  pattern  space.  This  makes  it  possible  to 
provide  the  distance  of  individuals  from  both  the  group  prototype  points  and 
the  decision  surface  separating  groups.  The  distance  between  an  individual 
and  the  decision  surface  separating  that  person's  group  from  another 
reflects  the  degree  to  which  that  individual  belongs  to  the  group.  Large 
distances  from  the  decision  surface  would  reflect  strong  identification  of 
the  individual  with  the  group.  The  closer  the  individual  is  to  the  decision 
surface,  the  more  similar  that  person  is  to  the  other  group.  Negative 
distances  from  the  decision  surface  indicate  that  the  individual  is  on  the 
wrong  side  of  the  decision  surface  and  therefore,  misclassified .  The 
distance  between  an  individual  and  the  class  prototype  indicates  the  degree 
to  which  that  person  represents  the  average  features  of  that  class. 

The  distances  from  an  individual  to  both  the  decision  surface  and  the 
group  prototypes  for  the  network  solutions  of  all  individuals  are  given  in 
Tables  3  through  7.  Information  based  on  classification  of  ratings  and  MDS 
patterns  is  available  in  an  earlier  report  (Schvaneveldt  et  al.,  1982).  It 
should  be  noted  that  there  are  no  negative  distances  from  any  individual  to 
the  decision  surfaces  for  each  of  the  network  classifications;  consequently, 
there  were  no  misclassi fications  of  individuals  from  any  of  the  groups. 

It  appears  that,  even  without  a  metric  for  link  length,  the  network 
solutions  still  provide  enough  structural  information  to  separate  groups 
with  a  great  deal  of  accuracy.  This  is  indicated  by  the  100  percent  correct 
classification  of  all  individuals  in  their  respective  groups.  On  the 
average,  individuals  also  were  closest  to  their  own  group  prototypes 
compared  to  the  prototypes  cf  the  other  groups.  The  average  distance  from 
the  class  prototype  to  each  class  member  given  in  Tables  3  through  7 
provides  a  measure  for  the  degree  of  class  clustering.  Shorter  distances 
suggest  more  homogeneous  classes  with  greater  consistency  in  the  individual 
conceptual  structures.  For  each  group  this  measure  is  found  in  the  table 
corresponding  to  that  group's  individual  distances.  It  is  the  average 
distance  from  the  individuals  to  their  own  group  prototype  and  is  the  value 
underlined  in  the  tables.  For  both  raanuevers  it  was  found  that  IPs  cluster 
most  tightly  while  UPs  are  the  most  variable.  This  seems  logical 
considering  that  IPs  follow  standardized  procedures  for  presenting  the 
maneuvers  and  have  probably  developed  similar  ways  of  thinking  about  them. 
The  UPs,  on  the  other  hand,  are  still  learning  the  material  and  have 
various  ways  of  thinking  about  the  relationships  among  these  concepts. 
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Table  3 


Separation  of  IPs  from  Other  Groups 
Based  on  a  Minimum-Distance  Classifier 
of  Network  Solutions 

Split-Plane  Maneuvers 

II  through  17  are  individual  IPs 


Distances  from  Distances  from 

Decision  Surface  Group  Prototypes 


GP 

IW 

UP 

IP 

GP 

IW 

UP 

11 

1.71 

2.48 

1.86 

6.86 

7.84 

8.51 

8.09 

12 

2.42 

2.75 

2.80 

7.26 

8.56 

9.00 

8.98 

13 

2.28 

2.94 

2.80 

7.16 

8.40 

9.03 

8.90 

14 

1.72 

2.69 

1.97 

7.45 

8.37 

9.11 

8.67 

15 

1.39 

2.30 

1.80 

7.27 

8.30 

8.75 

8.41 

16 

2.40 

2.32 

3.28 

6.54 

7.94 

8.15 

8.68 

17 

2.44 

2.48 

2.92 

8.00 

9.20 

9.46 

9.65 

Average 

from 

group  prototypes: 

7.22 

8.37 

8.86 

8.77 

Low  Angle 

11  through  16  are 

Distances  from 
Decision  Surface 

Strafe 

individual  IPs 

Distances  from 
Group  Prototypes 

IW 

UP 

IP 

IW 

UP 

11 

2753 

2.82 

6.56 

8.13 

8.49 

12 

2.83 

2.47 

7.19 

8.75 

8.78 

13 

1.65 

2.31 

6.24 

7.32 

7.93 

14 

2.53 

2.44 

6.53 

8.06 

8.24 

15 

1.73 

2.20 

7.37 

8.34 

8.77 

16 

1.85 

3.22 

7.66 

8.66 

9.58 

Average 

from  group  prototype: 

6.92 

8.21 

8.63 

*  Each  row  of  the  table  represents  one  IP.  Decision  surfaces  were 
computed  for  separating  IPs  from  the  three  remaining  groups.  The 
distance  from  each  IP  to  a  decision  surface  is  shown  along  with  the 
distance  from  each  IP  to  each  group  prototype.  Misclassifications 
would  be  indicated  by  negative  distances  to  the  decision  surface; 
however,  there  were  no  misclassifications  in  this  case. 
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Table  4 


Separation  of  GPs  from  Other  Groups 
Based  on  a  Minimum-Distance  Classifier 
of  Network  Solutions 

Split-Plane  Maneuvers 

G1  through  G9  are  individual  GPs 


Distances  from  Distances  from 

Decision  Surface  _ Group  Prototypes 


IP 

IW 

UP 

IP 

GP 

IW 

UP 

G 1 

2.02 

2.03 

1.98 

8.56 

7.49 

8.86 

8.57 

G2 

1.86 

2.73 

2.29 

8.43 

7.44 

9.25 

8.68 

G3 

1.15 

1.96 

1.58 

7.82 

7.17 

8.54 

8.07 

G4 

2.21 

3.59 

1.84 

8.87 

7.74 

9.97 

8.72 

G5 

2.50 

3.08 

1.75 

9.32 

8.10 

9.97 

8.99 

G6 

3.37 

3.83 

2.51 

9.58 

7.95 

10.27 

9.23 

G7 

1.90 

3.12 

2.79 

8.11 

7.04 

9.17 

8.60 

G8 

2.86 

2.57 

2.29 

9.56 

8.20 

9.77 

9.33 

G9 

1.24 

1.86 

2.55 

8.69 

8.07 

9.25 

9.34 

Average 

from  group  prototypes: 

8.77 

7.69 

9.45 

8.84 

*  Each  row  of  the  table  represents  one  GP.  Decision  surfaces  were 
computed  for  separating  GPs  from  the  three  remaining  groups.  The 
distance  from  each  GP  to  a  decision  surface  is  shown  along  with  the 
distance  from  each  GP  to  each  group  prototype.  Misclassifications 
would  be  indicated  by  negative  distances  to  the  decision  surface; 
however,  there  were  no  misclassifications  in  this  case. 


Table  5 


Separation  of  IWs  from  Other  Groups 
Based  on  a  Min imun-Di stance  Classifier 
of  Network  Solutions 

Split-Plane  Maneuvers 

W1  through  W4  are  individual  IWs 


Distances  from  Distances  from 

Decision  Surface  _ Group  Prototypes 


IP 

GP 

UP 

IP 

GP 

IW 

UP 

W1 

2.91 

3.02 

3.20 

9.16 

9. 34 

7.35 

9.52 

W2 

4. 17 

3.74 

3.36 

10.36 

10.28 

8.03 

10. 14 

W3 

1.75 

2.45 

2.23 

7.94 

8.48 

6.71 

8.40 

W4 

1.44 

1.81 

2.66 

7.83 

8.15 

6.82 

8.77 

Average 

from  group  prototypes: 

8.82 

9.06 

7.23 

9.21 

Low  Angle  Strafe 

W1  through  W7  are  individual  INs 


Distances  from  Distances  from 

Decision  Surface  Group  Prototypes 


IP 

UP 

IP 

IW 

UP 

W1 

3.01 

3.34 

8.94 

7.31 

9.30 

W2 

2.00 

2.63 

7.91 

6.71 

8.42 

W3 

1.19 

2.18 

7.79 

7.08 

8.47 

W4 

1.30 

1.52 

8.12 

7.39 

8.34 

W5 

3.82 

2.41 

10.58 

8.85 

10.10 

W6 

1.74 

1.83 

8.14 

7.14 

8.31 

W7 

2.36 

3.40 

8.46 

7.13 

9.19 

Average 

from  group  prototypes: 

8.56 

7.37 

8.88 

*  Each  row  of  the  table  represents  one  IW.  Decision  surfaces  were 
computed  for  separating  IWs  from  the  three  remaining  groups.  The 
distance  from  each  IW  to  a  decision  surface  is  shown  along  with  the 
distance  from  each  IW  to  each  group  prototype.  Misclassifications 
would  be  indicated  by  negative  distances  to  the  decision  surface; 
however,  there  were  no  misclassifications  in  this  case. 
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Table  6 


Separation  of  UPs  from  Other  Groups 
Based  on  a  Minimum-Distance  Classifier 
of  Network  Solutions 

Split-Plane  Maneuvers 


U1  through  U17  are  individual  UPs 


Distances  from  Distances  from 

Decision  Surface  _ Group  Prototypes 


IP 

GP 

IW 

IP 

GP 

IW 

UP 

U1 

3.09 

2.63 

2.25 

9.52 

9.09 

9.25 

7.73 

U2 

2.59 

1.95 

3.41 

10.13 

9.69 

10.77 

8.77 

U3 

2.10 

1.89 

3.28 

11.33 

11.13 

12.04 

10.37 

U4 

2.75 

2.20 

2.94 

9.17 

8.71 

9.51 

7.54 

U5 

2.27 

1.83 

3.37 

8.69 

8.30 

9.57 

7.27 

U6 

2.95 

2.63 

2.99 

10.96 

10.66 

11.18 

9.53 

U7 

2.70 

2.15 

2.02 

9.52 

9.09 

9.33 

7.99 

U8 

2.18 

1.77 

2.59 

8.75 

8.38 

9.19 

7.41 

U9 

2.54 

2.09 

3.21 

8.60 

8.18 

9.25 

6.98 

U10 

2.27 

2.13 

2.98 

9.41 

9.19 

10.00 

8.12 

U11 

2.71 

2.51 

3.45 

9.03 

8.74 

9.70 

7.39 

U12 

2.31 

2.08 

2.78 

8.95 

8.68 

9.43 

7.56 

U 1 3 

2.65 

1.84 

2.36 

9.53 

8.97 

9.57 

8.03 

U 1 4 

1.31 

2.15 

2.03 

7.86 

8.22 

8.49 

6.99 

U 15 

2.84 

2.06 

3.06 

9.99 

9.45 

10.32 

8.45 

U16 

1.52 

2.25 

2.59 

9.33 

9.57 

10.07 

8.48 

U17 

3.56 

2.84 

3.33 

11.19 

10.71 

11.31 

9.48 

Average 

from  i 

group  prototypes: 

9.53 

9.22 

9.94 

8.12 

*  Each  row  of  the  table  represents  one  UP.  Decision  surfaces  were 
computed  for  separating  UPs  from  the  three  remaining  groups.  The 
distance  from  each  UP  to  a  decision  surface  is  shown  along  with  the 
distance  from  each  UP  to  each  group  prototype.  Misclassifications 
would  be  indicated  by  negative  distances  to  the  decision  surface; 
however,  there  were  no  misclassifications  in  this  case. 
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Table  7 


Separation  of  UPs  from  Other  Groups 
Based  on  a  Minimum-Distance  Classifier 
of  Network  Solutions 

Low  Angle  Strafe 

U 1  through  U 1 6  are  individual  UPs 


Distances  from  Distances  from 

Decision  Surface  Group  Prototypes 


IP 

IW 

IP 

IW 

UP 

U1 

4.06 

3.90 

12.50 

12.37 

10.70 

U2 

2.88 

3.60 

10.68 

10.95 

9.18 

U3 

2.92 

2.43 

9.49 

9.16 

7.73 

U4 

1.74 

1.74 

8.79 

3.75 

7.70 

U5 

3.39 

3.36 

12.35 

12.08 

10.61 

Jo 

1.68 

1.26 

9.16 

8.90 

8.17 

U7 

2.00 

1.84 

8.77 

8.63 

7.51 

U8 

3.01 

2.86 

10.13 

9.99 

8.46 

U9 

1.72 

2.19 

8.96 

9.18 

7.91 

U10 

2.96 

3.23 

10.47 

10.53 

8.89 

un 

2.94 

2.79 

9.64 

9.50 

7.92 

U12 

1.87 

1.59 

9.02 

8.82 

7.88 

U 1 3 

1.71 

1.69 

8.74 

8.68 

7.66 

U14 

1.70 

1.91 

8.85 

8.93 

7.80 

U15 

2.17 

1.97 

9.56 

9.40 

8.30 

U16 

4.00 

3.21 

11.57 

11.15 

9.63 

Average 

from  group  prototypes: 

9.92 

9.81 

8.50 

•  Each  row  of  the  table  represents  one  UP.  Decision  surfaces  were 
computed  for  separating  UPs  from  the  three  remaining  groups.  The 
distance  from  each  UP  to  a  decision  surface  is  shown  along  with,  the 
distance  from  each  UP  to  each  group  prototype.  Misclassifications 
would  be  indicated  by  negative  distances  to  the  decision  surface; 
however,  there  were  no  misclassifications  in  this  case. 


The  classification  data  discussed  so  far  have  resulted  from  applying 
discriminant  functions  to  members  of  classes  from  which  the  discriminant 
functions  were  originally  derived.  Although  this  provides  useful  inform¬ 
ation  about  class  and  individual  differences,  it  is  not  a  direct  test  of  the 
ability  of  discriminant  functions  to  categorize  members  of  unknown  classes. 
The  second  analysis  involved  generating  a  discriminant  function  on  the  basis 
of  a  limited  training  set  from  two  classes  and  then  using  the  function  to 
place  new  and  unknown  members  into  one  of  the  two  classes. 

The  results  of  this  analysis  are  given  in  Tables  8  and  9  for  the  split- 
plane  maneuvers  and  Table  10  for  the  low  angle  strafe  maneuver.  The  tables 
give  the  total  number  of  individuals  for  which  classification  was  attempted, 
followed  by  the  percentage  of  those  correctly  classified  for  the  ratings, 
MDS,  and  network  patterns.  Since  100  different  randomly  chosen  training 
sets  were  used  for  each  training  set  size,  the  number  of  classifications 
attempted  is  always  100  times  the  number  of  remaining  members  in  the  two 
classes.  The  probability  of  randomly  classifying  correctly  at  least  the 
obtained  number  of  correct  classifications  for  each  training  set  size  was 
less  than  .01  for  all  but  two  instances  for  the  split-plane  maneuver.  The 
two  exceptions  had  a  probability  less  than  .05  and  occurred  when  the  train¬ 
ing  set  size  was  1  and  1.  This  probability  is  associated  with  the  comparison 
of  IPs  and  GPs  and  the  comparison  of  IPs  and  IWs.  For  the  low  angle  strafe 
maneuver  the  probabilities  were  less  than  .01  for  the  comparison  of  IPs 
and  UPs  and  the  comparison  of  IWs  and  UPs.  For  the  comparison  of  IPs  and 
IWs,  probabilities  were  less  than  .05  for  all  training  set  sizes,  except 
when  the  training  set  size  was  1  and  1,  in  which  case  the  probability  was 
less  than  .10. 

Pattern  classification  with  networks  was  successful.  Table  8  shows  that 
with  only  one  member  each  from  the  IPs  and  GPs  on  which  to  base  a  decision 
surface,  742  out  of  1400  remaining  IPs  were  classified  correctly.  With  only 
three  members  each  from  the  IPs  and  UPs  it  is  possible  to  classify  correctly 
the  remaining  18  members  with  an  88  percent  accuracy  rate.  Table  10  shows 
that  the  classification  was  poorest  for  IPs  and  IWs  with  the  low  angle 
strafe  maneuver.  Table  2  shows  that  the  distance  between  group  prototypes 
is  shortest  for  IPs  and  IWs,  indicating  that  they  are  the  most  similar  pair 
of  groups  in  viewing  this  maneuver.  This  points  out  that,  in  general,  class¬ 
ification  improves  as  the  distance  between  classes  in  the  pattern  space 
increases. 

Classification  generally  improves  also  as  the  size  of  the  training  set 
increases.  With  larger  training  sets,  the  discriminant  function  is  derived 
from  a  larger,  more  representative  sample.  Exceptions  to  this  sometimes 
occur  when  a  few  members  in  one  class  strongly  resemble  members  of  the  other 
class.  In  this  case,  the  decrease  in  percent  correct  is  probably  due  to 
the  difficulty  in  classifying  these  outliers. 
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Table  8 


Classification  of  Group  Members  Using  Limited  Training  Sets 


Split-Plane  Maneuver  Concepts 


Percent  Correct  of  Classifications 


Training 
Set  Size 

Number 

Classified  Ratings 

Distance 
'  n  MDS 

Network 

IPs 

GPs 

1 

1 

1400 

57 

68 

53 

2 

2 

1200 

67 

77 

56 

3 

3 

1000 

67 

82 

60 

4 

4 

800 

68 

84 

62 

5 

5 

600 

62 

87 

64 

6 

6 

400 

52 

87 

64 

7 

7 

200 

51 

84 

66 

Average:  63 

79 

61 

Training 
Set  Size 

Percent  Correct  of  Classifications 

Number 

Classified 

Ratings 

Distance 
in  MDS 

Network 

IPs 

IWs 

1 

1 

900 

53 

56 

53 

2 

2 

700 

57 

60 

61 

3 

3 

500 

61 

63 

64 

4 

4 

300 

71 

67 

72 

Average 

:  58 

60 

63 

Training 
Set  Size 

Percent  Correct  of  Classifications 

Number 

Classified 

Ratings 

Distance 
in  MDS 

Network 

IPs 

UPs 

1 

1 

2200 

65 

79 

70 

2 

2 

2000 

77 

95 

80 

3 

3 

1800 

78 

98 

88 

4 

4 

1600 

83 

100 

39 

5 

5 

1400 

89 

100 

91 

6 

6 

1200 

91 

100 

94 

7 

7 

1000 

96 

100 

94 

Average:  80 

94 

87 
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Table  9 


Classification  of  Group  Members  Using  Limited  Training  Sets 


Training 
Set  Size 

Spli 

Number 

Classifie 

t-Plane  Maneuver  Concepts 

Percent  Correct  of  Classifications 

d  Ratings 

Distance 
in  MDS 

Network 

GPs 

IWs 

1 

1 

1100 

58 

62 

62 

2 

2 

900 

64 

71 

70 

3 

3 

700 

65 

76 

74 

4 

4 

500 

70 

76 

74 

Average:  63 

70 

70 

Training 
Set  Size 

Percent  Correct  of  Classifications 

Number 

Classified 

Ratings 

Distance 
in  MDS 

Network 

GPs 

UPs 

1 

1 

2400 

62 

79 

67 

2 

2 

2200 

70 

89 

76 

3 

3 

2000 

74 

93 

82 

4 

4 

1800 

76 

95 

89 

5 

5 

1600 

76 

95 

90 

6 

6 

1400 

76 

95 

94 

7 

7 

1200 

73 

95 

95 

8 

8 

1000 

73 

96 

96 

9 

9 

800 

72 

98 

97 

Average:  72 

92 

87 

Percent  Correct  of  Classifications 

Training 
Set  Size 

Number 

Classified  Ratings 

Distance 
in  MDS 

Network 

UPs  IWs 

1  1 

1900 

68 

76 

65 

2  2 

1700 

81 

93 

84 

3  3 

1500 

81 

94 

90 

4  4 

1300 

78 

95 

94 

Average:  76 

89 

83 
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Table  10 


Classification  of  Group  Members  Using  Limited  Training  Sets 

Low  Angle  Strafe 


Percent  Correct  of  Classifications 


Training 
Set  Size 

Number 

Classified 

Ratings 

Distance 
in  MDS 

Network 

IPs 

IWs 

1 

1 

1100 

49 

56 

48 

2 

2 

900 

43 

54 

51 

3 

3 

700 

45 

53 

55 

4 

4 

500 

47 

55 

55 

5 

5 

300 

43 

53 

57 

6 

6 

100 

56 

56 

60 

Aver  age 

:  46 

55 

54 

Training 
Set  Size 

Percent  Correct  of  Classifications 

Number 

Classified 

Ratings 

Distance 
in  MDS 

Network 

IPs 

UPs 

1 

1 

2000 

59 

84 

59 

2 

2 

1800 

72 

96 

67 

3 

3 

1600 

73 

96 

73 

4 

4 

1400 

81 

97 

76 

5 

5 

1200 

79 

97 

79 

6 

6 

1000 

30 

95 

86 

Average 

:  73 

94 

73 

Training 
Set  Size 

Percent  Correct  of  Classifications 

Number 

Classified 

Ratings 

Distance 
in  MDS 

Network 

UPs 

IWs 

1 

1 

2100 

58 

69 

S3 

2 

2 

1900 

62 

38 

71 

3 

3 

1700 

65 

91 

74 

4 

4 

1500 

67 

92 

80 

5 

5 

1300 

67 

93 

78 

6 

6 

1100 

60 

93 

86 

7 

7 

900 

54 

87 

85 

Average: 

62 

87 

77 
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The  discriminant  functions  based  on  the  network  patterns  resulted  in 
better  classification  of  unknown  members  compared  to  the  rating  patterns. 
The  average  percentage  of  correct  classifications  for  the  network  patterns 
was  significantly  greater  than  the  average  for  the  ratings  (p<.01,  sign- 
test,  based  on  the  binomial  distribution).  Apparently,  the  structural 
information  captured  by  the  networks  is  superior  to  the  information  in  the 
rating  data  despite  the  simplicity  of  the  network  representation.  Only  the 
presence  or  absence  of  links  was  used  in  the  patterns.  The  MD3  patterns 
resulted  in  better  classification  of  unknown  members  compared  with  the 
network  patterns.  The  average  percentage  of  correct  classifications  for  the 
MDS  pattern  was  significantly  greater  than  the  average  for  the  network 
patterns  (pC.01,  sign-test).  The  superiority  of  MDS  may  be  due  to  the 
metric  information  it  provides.  Further  work  is  required  to  determine 
whether  adding  metric  information  to  the  networks  will  improve  their  ability 
to  classify  individuals. 

The  goal  of  this  phase  of  the  project  has  been  to  test  the  ability  of 
network  representations  to  discriminate  between  individuals  in  different 
groups.  In  general,  the  pattern  classification  techniques  appear  to  provide 
a  sensitive  method  for  detecting  differences  between  the  networks  of  both 
groups  and  individuals.  The  finding  that  the  distance  between  concepts  in 
an  MDS  solution  allow  for  more  accurate  classification  than  does  the  network 
pattern  illustrates  the  need  for  deriving  a  metric  which  describes  the 
length  of  network  links.  It  would  be  of  interest  to  examine  the  results  of 
the  classification  of  a  pattern  containing  the  metric  for  the  length  of 
links  in  a  network  relative  to  the  classification  of  the  MDS  distances. 


Identification  of  Links  between  Concepts 


As  seen  in  several  applications  of  the  network  algorithm,  networks 
provide  useful  and  inter  pretable  structures.  The  algorithm  itself,  however  , 
only  indicates  where  the  links  are  in  a  set  of  concepts.  The  task  of 
identifying  the  nature  of  the  links  is  the  purpose  of  the  present  study.  An 
expert  fighter  pilot  assisted  in  identifying  the  nature  of  the  relation 
represented  by  the  links  in  the  networks  of  IPs  for  split  plane  maneuvers 
and  the  low  angle  strafe  (Schvaneveldt  et  al.,  1982).  The  results  of  this 
analysis  are  presented  in  Tables  11  and  12.  It  should  be  noted  that  there 
were  many  fewer  types  of  links  than  there  were  links. 

For  the  split-plane  maneuvers,  there  were  seven  types  of  links:  AFFECTS 
(15  links),  IS  A  (11  links),  LEADS  TO  (5  links),  DESIRABLE  (4  links), 
ACCEPTABLE  (2  links),  SELECTS  (2  links),  and  INSTRUMENT  OF  (1  link).  In 
some  cases,  a  more  general  relation  is  used  where  more  specific  relations 
could  be  specified.  In  particular,  the  general  relation  AFFECTS  can  be  made 
more  specific,  for  some  of  the  links.  For  example,  while  quarter  plane 
affects  angle  off,  the  specific  purpose  of  the  quarter  plane  is  to  INCREASE 
angle  off.  As  an  initial  classification  attempt,  however,  the  fewest 
possible  types  of  relations  were  used.  The  general  meaning  of  the  relations 
i  s  as  follows.  Let  "first"  represent  the  first  element  of  a  linked  pair  of 
concepts  and  "second"  represent  the  second  element.  AFFECTS  means  that 
first  leads  to  some  change  in  second.  IS  A  means  that  first  is  one  member  of 
the  category  designated  by  second.  LEADS  TO  means  that  first  produces  the 
result  designated  by  second.  DESIRABLE  means  that  first  is  an  optimal  or 
desirable  condition  for  second.  ACCEPTABLE  means  that  first  is  an 
acceptable  (but  not  optimal)  condition  for  second.  SELECTS  means  that  first 
is  involved  in  choosing  or  selecting  second.  INSTRUMENT  OF  means  that  first 
is  the  instrument  of  the  action  designated  by  second. 

For  the  low  angle  strafe,  there  were  also  seven  types  of  links:  AFFECTS 
(12  links),  DETERMINES  (11  links),  IS  POINT  OF  REFERENCE  FOR  (5  links), 
DESIRABLE  (5  links),  IS  (3  links),  AVOIDS  (2  links),  and  INSTRUMENT  OF  (1 
link)  .  Not  surprisingly,  some  of  the  relations  occur  in  both  sets  of 
concepts.  The  relations  AFFECTS,  DESIRABLE,  and  INSTRUMENT  OF  seem  to  be 
natural  relations  in  the  domain  of  fighter  maneuvers.  There  are  also 
various  ways  of  describing  categorical  relationships.  These  appear  as  IS  A 
in  the  split-plane  concepts  and  as  IS  POINT  OF  REFERENCE  FOR  and  ^S  in  the 
low  angle  strafe  concepts.  The  relation  DETERMINES  represents  a  stronger 
version  of  the  relation  AFFECTS.  DETERMINES  means  that  the  first  element 
leads  to  the  second,  regardless  of  other  factors.  AFFECTS  means  that  the 
first  element  can  lead  to  the  second  depending  on  other  factors. 

With  the  addition  of  information  about  the  identity  of  the  relations 
represented  by  links  in  the  network,  the  network  representation  becomes  a 
more  complete  representation  of  the  conceptual  structure  of  IPs.  As  such, 
it  may  be  useful  in  communicating  some  of  the  important  conceptual  relations 
to  trainees.  The  network  with  labelled  links  should  also  prove  useful  in 
further  attempts  to  define  conceptual  structure  and  to  relate  these 
structures  to  actual  performance  in  the  aircraft. 
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Table  11  -  Concept  LINKS  -  Split  Plane  Maneuvers 

_ AFFECTS _ 

low  yo  yo-acceleration 
quarter  plane-angle  off 

lag  roll-relative  energy 
lag  roll-aspect  angle 
barrel  roll-aspect  angle 
weapons  parameter s-guns 
weapons  parameters-heat 

airspeed-vertical  maneuvering 
airspeed-corner  velocity 
acceleration-G  loading 
radial  G-G  loading 
power  setting-smash 
power  setting-extension 
lift  vector-radial  G 
lift  vector-vertical  maneuvering 

_ IS  A _ 

quarter  plane-vertical  maneuvering 
barrel  roll-vertical  maneuvering 
c  uto  f  f-ov  er  ta  ke 
smash-over take 
airspeed-acceleration 
smash-air  speed 

G  loading-weapons  parameters 
smash-relative  energy 
3-9  line-aspect  angle 
aspect  angle-weapons  parameters 
high  yo  yo-quarter  plane 

_ LEADS  TO _ 

quarter  plane-lag  pursuit 
lag  roll-lag  pursuit 
angle  off-snapshot 
lead  pursuit-cutoff 
acceleration-extension 

_ DESIRABLE _ 

lead  pursuit-guns 

airspeed-pure  pursuit 
6  O'Clock-heat 
3-9  line-quarter  plane 

_ ACCEPTABLE _ 

lag  pursuit-heat 
pure  pursuit-heat 

_ SELECTS _ 

s wi tc  hoi ogy-g  un  s 
s wi tc  hoi og  y-he  at 

_ INSTRUMENT  OF _ 

guns-snapshot 
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Table  12  -  Concept  LINKS  -  Low  Angle  Strafe 


_ AFFECTS _ 

drift-bullet  impact 
dive  angle-recovery 

bunt-glide  path 
airspeed-closure 
closure-foul 
bank-tracking 
pipper  fixation-foul 

pull  up-walking 
yaw-tracking 
range-bullet  impact 
pipper  placement-bullet  impact 
pipper  placement-aim  point 


IS  POINT  OF  REFERENCE  FOR 
foul  line-recovery 
foul  line-fire 
run  in  line-final 

aim  point-pipper  fixation 
final-tracking 


_ IS 

glide  path-final 
burst-  fire 
foul  line-range 


INSTRUMENT  OF 
gun  s- fire 


_ DETERMINES _ 

bank-drift 

aim  off  point-glide  path 
aim  off  point-pipper  placement 
dive  angle-glide  path 
foul  line-foul 
altitude- foul 
walking-bullet  impact 
trigger-fire 

pipper  placement-tracking 

bunt-pi pper  placement 
pull  up-recovery 


DESIRABLE 
stab ilize-air speed 
stabilize-tr im 
stabili ze-trac  king 
tr ac king-fire 


AVOIDS _ 

pull  up-ricochet 
pull  up- foul 
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MULTIDIMENSIONAL  SCALING 


Multidimensional  scaling  is  a  powerful  technique  for  producing  struct¬ 
ural  descriptions  of  empirical  similarity  judgments.  The  technique  arranges 
the  judged  concepts  in  an  N-dimensional  space  where  the  Euclidean  distances 
between  points  reflect  the  psychological  proximity  of  the  concepts.  The  MDS 
supplies  several  important  pieces  of  information.  First,  it  summarizes  the 
data  into  a  spatial  configuration,  which  though  at  times  complex,  is 
considerably  more  informative  than  are  the  empirical  similarity  judgments. 
Second  ,  MDS  captures  the  global  relations  among  the  concepts.  That  is,  MDS 
considers  the  relationship  of  each  concept  to  all  other  concepts  and  places 
the  concepts  along  the  dimensions  of  the  space  in  a  way  that  reflects  these 
relations.  Although  such  a  procedure  can  distort  local  relationships,  the 
procedure  is  unsurpassed  at  revealing  global  structure.  While  identifying 
the  dimensions  of  the  space  may,  at  times,  prove  problematic,  successful 
identification  supplies  information  about  conceptual  structure  that  cannot 
be  gleaned  from  the  original  ratings  nor  from  other  scaling  techniques. 
Finally,  MDS  supplies  a  metric  (distance  between  concepts  in  multidimen¬ 
sional  space)  that  has  some  useful  applications. 

Optimal  Dimensionality  in  Representing  Concepts 

Most  MDS  programs  require  the  user  to  specify  the  number  of  dimensions 
to  be  used  in  scaling  the  input  data.  In  representing  a  specific  conceptual 
structure,  determining  the  number  of  dimensions  that  best  describes  the 
conceptual  space  may  be  somewhat  problematic.  The  use  of  too  few  dimensions 
will  result  in  the  loss  of  potentially  important  information  about  the 
interrelationships  among  concepts.  In  contrast,  uncovering  the  relation¬ 
ships  between  concepts  becomes  increasingly  complex  as  the  number  of 
dimensions  used  to  describe  the  conceptual  structure  increases.  Thus,  using 
too  many  dimensions  will  result  in  an  unnecessarily  complex  representation 
of  the  conceptual  structure.  The  goal  then  is  to  use  the  minimum  number  of 
dimensions  that  will  preserve  meaningful  relationships  between  concepts  in 
representations  of  a  conceptual  space. 

The  earlier  approach  (Schvaneveldt  et  al . ,  1982)  to  determining  the 
appropriate  number  of  dimensions  was  to  increase  systematically  the  number 
of  dimensions  in  the  scaling  program  and  to  plot  the  amount  of  variance  in 
the  IPs  data  which  was  accounted  for  by  MDS  solutions.  The  point  at  which 
the  function  began  to  level  off  was  selected  as  the  appropriate  dimension¬ 
ality  for  that  set  of  concepts.  For  the  split  plane  maneuvers,  a  five¬ 
dimensional  solution  accounted  for  80  percent  of  the  variance  in  the  input 
data  (similarity  ratings)  and  was  selected  as  the  appropriate  dimensionality 
for  the  3plit-plane  data.  Two  of  these  dimensions  were  associated  with 
temporal  factors,  and  a  third  dimension  with  distinguishing  maneuvers 
appropriate  with  lead  pursuit  as  opposed  to  lag  pursuit.  The  other  two 
dimensions  have  not  yet  been  Identified. 
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It  was  discovered  that  the  distances  recovered  from  MDS  scaling 
solutions  were  better  at  discriminating  between  individuals  in  different 
groups  (IPs,  GPs,  UPs,  and  IWs)  than  were  the  rating  data  on  which  the  MDS 
solutions  were  based.  This  result  led  to  reexamination  of  the  issue  of 
optimal  dimensionality  from  a  different  perspective.  If  MDS  distances  are 
superior  to  ratings  and  the  two  correlate  highly,  perhaps  the  superiority  of 
MDS  was  constrained  by  using  as  many  as  five  dimensions.  Since  the 
correlations  between  MDS  distances  and  ratings  necessarily  increase  as  the 
number  of  dimensions  increases,  possibly  MDS  distances  were  been 
artificially  constrained  to  correspond  to  the  ratings.  To  verify  that  the 
conceptual  space  for  split  plane  maneuvers  is  best  represented  with  a 
minimum  of  five  dimensions,  conceptual  structures  derived  from  multi¬ 
dimensional  scaling  output  using  two,  three,  and  four  dimensions  were 
compared  with  the  five-dimensional  structures. 

Table  13  shows  the  average  correlations  between  the  MDS  distances  for 
pairs  of  individual  subjects.  The  distances  were  derived  from  multi¬ 
dimensional  scaling  using  two,  three,  four,  and  five  dimensions.  Notice  that 
the  correlations  obtained  using  four  dimensions  are  comparable  to  those 
obtained  using  five  dimensions,  and  even  slightly  higher  in  some  cases.  The 
correlations  within  and  between  the  IP  and  GP  groups  using  three  dimensions 
are  also  quite  similar  to  the  four-  and  five-dimensional  correlations. 
Apparently,  little  information  about  the  conceptual  structure  of  these 
concepts  is  lost  when  dropping  one  and  possibly  two  dimensions  from  the 
conceptual  3pace.  The  two-dimensional  correlations  are  somewhat  lower,  and 
in  fact,  are  comparable  to  those  obtained  using  the  raw  data,  or  rating 
scores. 

Perhaps  a  more  salient  indication  of  the  importance  in  the  way  this 
conceptual  space  is  represented  is  the  ability  of  pattern  recognition 
techniques  to  discriminate  between  groups  of  flying  personnel  based  on 
conceptual  structures  represented  in  two,  three,  four,  and  five  dimensions. 
Four  groups  were  examined:  GPs,  IPs,  IWs  and  UPs.  Each  individual  is 
represented  as  a  distance  vector  consisting  of  the  435  distances  between 
concepts  (  30  concepts  taken  two  at-a-time)  in  the  MDS  spatial  structure.  The 
classification  of  individuals  into  groups  will  be  more  successful  when  the 
groups  are  more  highly  separable.  Further,  assuming  that  the  number  of 
dimensions  used  to  represent  a  conceptual  space  has  some  effect  on  the 
degree  of  group  differentiation,  the  optimal  number  of  dimensions  used  to 
represent  the  conceptual  space  is  the  number  at  which  groups  of  flying 
personnel  are  most  highly  separable. 
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Table  13 


Average  Correlations  Within  Groups  and  Between  Groups 
for  Split-Plane  Maneuvers 


Two  Dimensions  Three  Dimensions 


IP 

GP 

IP  GP 

IP 

.42 

.34 

IP 

.45  .39 

GP 

.34 

.36 

GP 

.39  .42 

Four  Dimensions 

IP 

GP 

IW 

UP 

Aver age 

IP 

.46 

.38 

.41 

.22 

.37 

GP 

.38 

.40 

.34 

.24 

.34 

IW 

.41 

.34 

.37 

.20 

.33 

UP 

.22 

.24 

.20 

.30 

.24 

Five  Dimensions 

IP 

GP 

IW 

up 

Aver  age 

IP 

.45 

•  37 

.40 

.22 

.36 

GP 

.37 

•  39 

.34 

.24 

.34 

IW 

.40 

.34 

.38 

.20 

.33 

UP 

.22 

.24 

.20 

.29 

.24 
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Table  14  compares  the  performance  of  a  training  algorithm  in  discrim¬ 
inating  between  IPs  and  GPs.  The  training  algorithm  takes  some  number  of 
individuals  (shown  in  the  left-most  colunns)  from  each  of  two  groups, 
develops  a  classifier  that  sucessfully  discriminates  between  those 
individuals  in  the  two  groups,  and  then  applies  the  classifier  to  the  other 
members  of  the  groups.  The  percent  correct  assignments  are  shown  for  100 
iterations  for  each  size  of  initial  group.  Each  iteration  randomly  selects 
the  initial  individuals.  Classification  performance  using  distances  derived 
from  four-dimensional  solutions  compared  well  with  performance  using 
distances  derived  from  five-dimensional  solutions.  Slightly  poorer  perform¬ 
ance  occurred  with  two-  and  three-dimensional  distances. 


Table  14 

Training  Algorithm  Classification  Performance  of  IPs  and  GPs 
Using  Distances  Derived  From  Two,  Three,  Four,  and  Five  Dimensions 

Training 

Set  Size  Percent  Correct 


IP 

GP 

2D 

3D 

4D 

5D 

1 

1 

61 

62 

70 

68 

2 

2 

68 

70 

75 

77 

3 

3 

69 

76 

80 

82 

4 

4 

72 

79 

83 

84 

5 

5 

73 

82 

85 

87 

6 

6 

80 

84 

84 

87 

7 

7 

85 

80 

81 

84 

Mean 

69 

73 

78 

79 

Further  comparisons  involving  all  other  groups  of  pilots  (Table  15) 
revealed  that,  in  general,  the  training  algorithm  can  discriminate  between 
groups  equally  well  with  MDS  solutions  with  four  or  five  dimensions.  These 
comparisons  suggest  that  little  is  gained  from  increasing  the  conceptual 
3pac.e  from  four  to  five  dimensions  and  that  some  information  is  lost  by 
confining  the  space  to  less  than  four  dimensions. 


Table  15  -  Training  Algorithm  Classification  Performance 


Training 

Percent 

Training 

Percent 

Set 

Size 

Correct 

Set 

Si  ze 

Correct 

IP 

IW 

4D 

5D 

GP 

IW 

4D 

5D 

1 

1 

54 

56 

1 

1 

65 

62 

2 

2 

61 

60 

2 

2 

72 

71 

3 

3 

64 

63 

3 

3 

79 

76 

4 

4 

71 

67 

4 

4 

80 

76 

Mean 

60 

60 

Mean 

72 

70 

IP 

UP 

4D 

5D 

GP 

UP 

4D 

5D 

1 

1 

80 

79 

1 

1 

76 

79 

2 

2 

94 

95 

2 

2 

89 

89 

3 

3 

96 

98 

3 

3 

93 

93 

4 

4 

98 

100 

4 

4 

94 

95 

5 

5 

99 

100 

5 

5 

94 

95 

6 

6 

99 

100 

6 

6 

95 

95 

7 

7 

100 

100 

7 

7 

95 

95 

Mean 

94 

94 

8 

8 

94 

96 

9 

9 

95 

98 

Mean 

90 

92 

IW 

UP 

4D 

5D 

1 

1 

80 

76 

2 

2 

91 

93 

4D 

5D 

3 

3 

93 

94 

Grand  Mean 

87.3 

86.8 

4 

4 

93 

95 

Mean 

39 

89 
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Still  another  way  of  comparing  representations  of  concepts  using  various 
numbers  of  dimensions  can  be  obtained  from  a  minimum-distance  classifier. 
The  minimum-distance  classifier  is  a  pattern  recognition  technique  that 
calculates  a  prototype  for  each  group  by  averaging  feature  values  (distances 
between  concepts  in  this  case)  and  calculates  the  distance  between  each 
prototype  and  each  pattern.  Patterns  are  placed  into  the  group  that  is 
represented  by  the  closest  prototype.  Given  the  distances  between  patterns 
of  a  group  and  their  prototype,  and  the  distances  between  those  same 
patterns  and  the  prototypes  of  other  groups,  a  measure  of  group  differ¬ 
entiation  can  be  obtained.  For  example,  let  Dw  be  the  sum  of  the  distances 
of  patterns  within  a  group  to  the  group  prototype,  and  Db  be  the  sun  of  the 
distances  between  those  patterns  and  the  prototypes  of  contrasting  groups. 
The  ratio  Dw/Db  is  essentially  the  degree  to  which  patterns  of  a  group 
cluster  around  their  own  prototype,  divided  by  the  degree  to  which  those 
patterns  are  separated  from  prototypes  of  contrasting  groups.  Lower  ratios 
would  mean  that  patterns  of  the  same  group  are  close  to  their  prototype 
relative  to  prototypes  of  other  groups.  Consequently,  the  group  would  be 
highly  differentiated  from  other  groups.  Higher  ratios  would  mean  either 
that  patterns  within  a  group  are  far  away  from  their  group  prototype,  or 
that  patterns  within  a  group  are  relatively  close  to  prototypes  of 
contrasting  groups,  or  both.  The  average  ratio  for  a  set  of  groups  would 
give  an  indication  of  overall  group  differentiation. 

Table  16  shows  that  overall,  groups  were  most  differentiated  (had  lower 
structural  ratios)  when  represented  as  distances  derived  from  four¬ 
dimensional  scaling  solutions.  This  suggests  that  pattern  recognition 
techniques  should  be  able  to  classify  different  groups  of  pilots  more 
accurately  when  their  conceptual  structures  are  represented  as  distances 
derived  from  MDS  solutions  using  four  dimensions. 


Table  16 

Structural  Ratios  (Dw/Db)  of  Classes  of  Pilots  Using 
Two,  Three,  Four,  and  Five  Dimensions 


Class 

2D 

3D 

4D 

5D 

IP 

.803 

.815 

.769 

.772 

GP 

.836 

.836 

.802 

.810 

IW 

.761 

.759 

UP 

.783 

.797 

Mean 

.820 

.825 

.779 

.785 
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In  summary,  the  minimum  number  of  dimensions  that  would  optimally 
describe  a  conceptual  space  representing  split-plane  concepts  was  invest¬ 
igated.  Within  and  between  group  correlations  suggested  that  four  and 
possibly  three  dimensions  were  adequate  to  describe  the  conceptual  space. 
Other  measures,  however,  proved  to  be  more  highly  predictive  of  classif¬ 
ication  performance  using  pattern  recognition  techniques.  Structural  ratios, 
which  measure  the  amount  of  group  differentiation,  suggested  that  pattern 
recognition  techniques  would  perform  optimally  when  conceptual  structures 
were  represented  as  distances  derived  from  MDS  solutions  using  a  minimum  of 
four  dimensions.  This  was  verified  by  comparing  the  classification 
performance  of  a  training  algorithm  using  two,  three,  four,  and  five 
dimensions. 


Spatial  Representations  of  Individuals  and  Groups 

Previously  in  this  project,  MDS  techniques  have  been  used  to  derive 
spatial  representations  of  various  flight-related  concepts.  These  MDS 
representations  are  useful  in  that  they  (a)  illustrate  conceptual  relations 
in  a  concrete  manner,  (b)  depict  global  relations  and  dimensions  among  the 
concepts,  and  (c.)  produce  measures  of  spatial  distance  which  are  useful  in 
comparing  different  representations.  While  these  MDS  solutions  illustrate 
relations  among  concepts  in  an  individual's  cognitive  organization, 
information  about  the  relations  among  individuals  can  also  be  derived.  For 
instance,  within  and  between  group  correlations  of  these  solutions 
indicate  degree  of  agreement  between  and  within  groups  of  individuals. 
Pattern  classification  techniques  use  the  MDS  solution  to  compare  an 

individual  to  that  person's  group  in  terms  of  distance  from  decision 
surfaces  and  prototypes.  The  information  from  these  techniques,  however, 
lacks  the  concrete  spatial  representation  and  the  relational  and  dimensional 
information  of  the  MDS  solution.  Thus,  it  would  be  useful  to  use  scaling 

techniques  to  produce  a  spatial  representation  of  individuals,  just  as  was 
done  with  flight-related  concepts. 

The  purpose  of  this  phase  of  the  project  is  to  derive  MDS  represent¬ 
ations  of  individuals.  Not  only  should  these  representations  illustrate 
differences  between  and  within  groups  of  individuals,  but  they  should  also 
provide  an  indication  of  global  dimensions  that  separate  individuals  into 

these  groups.  This  dimensional  information  has  practical  applications  in 
terms  of  training  and  selection.  For  instance,  if  undergraduate  pilots  that 
were  closer  to  a  particular  end  of  a  dimension  were  also  more  likely  to 
succeed  as  fighter  pilots,  then  separation  of  individuals  along  this 

dimension  could  be  used  as  a  predictor  for  future  success.  Also,  the  closer 
that  an  individual  is  in  the  MDS  solution  to  one  particular  group  of  pilots, 
the  more  similar  that  person's  cognitive  representation  of  flight-related 
information  is  to  other  pilots  of  this  experience  level. 

There  are  several  ways  of  generating  MDS  solutions  for  individuals. 
McKeithen,  Reitman,  Rueter,  and  Hirtle  (1981)  investigated  the  differences 
among  computer  programmers  of  differing  levels  of  expertise.  Subjects 
recalled  a  list  of  ALGOL  W  reserved  words,  and  based  on  the  order  that  each 
subject  recalled  the  words,  a  tree  structure  was  generated.  The  number  of 
chunks  in  common  within  these  tree  structures  was  used  as  a  measure  of 
similarity  between  individuals.  These  similarity  values  were  then  arranged 
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in  a  matrix  and  scaled  using  MDS  techniques.  In  this  way,  individuals  were 
plotted  in  multidimensional  space.  Adelson  (1981),  in  a  similar  experiment 
with  computer  programmers,  derived  an  MDS  solution  for  individuals  by 
correlating  distance  matrices  for  each  subject.  From  these  correlations,  ar 
intersubject  correlation  matrix  was  constructed,  to  which  the  MDS  technique 
was  applied . 

An  additional  technique  that  seems  to  capture  ti=e  distance  information 
inherent  in  MDS  involves  the  calculation  of  distances  between  each  person 
using  the  distances  derived  from  their  individual  MDS  solutions.  In  this 
case,  the  individual  can  be  thought  of  as  a  point  in  "n-dimensional"  space 
(n  =  435  dimensions  based  on  the  435  distances  for  all  pairs  of  30  terms)  . 
The  distance  between  two  individuals  would  take  into  account  the  difference 
in  distance  for  each  of  the  435  pairs  of  points  for  the  two  individuals. 
This  "distance"  technique  will  be  used  in  this  project  to  derive  a  measure 
of  intersubject  similarity,  since  distance  measures  have  proven  useful  in 
pattern  recognition  analyses  of  individual  and  group  differences. 

Method 

Distance  vectors  produced  from  four-dimensional  MDS  solutions  of  both 
Split  and  Strafe  concepts  were  used  to  compare  individual  pilots.  Four 
dimensions  were  chosen  since  this  number  of  dimensions  has  been  found  to  be 
optimal,  as  mentioned  in  the  preceding  section  of  this  report.  Results  from 
split-plane  and  low  angle  strafe  concepts  were  analyzed  separately.  The 
distance  vectors  for  each  subject  consisted  of  435  distances  between  pairs 
of  30  concepts.  Thu3,  each  individual  pilot  can  be  thought  of  as  a  point  in 
435-dimensional  space. 

Distances  between  individuals  were  taken  as  the  Euclidean  distance 
between  the  points  representing  the  individuals  in  the  435-dimensional 
space.  These  distances  resulted  in  a  matrix  of  distances  with  individuals 
as  row3  and  columns.  These  distance  values  were  then  scaled  in  multidimen¬ 
sional  space  using  one  and  two  dimensions  for  both  the  split-plane  and 
strafe  concepts.  Also,  in  order  to  focus  on  differences  among  experts,  two- 
dimensional  representations  were  derived,  excluding  UPs. 

Results  and  Conclusions 

The  two-dimensional  solutions  for  all  subjects  can  be  seen  in  Figures  1 
and  2.  In  both  representations,  the  undergraduate  pilot  trainees  are 
linearly  separable  from  the  more  experienced  subjects  (IPs,  GPs,  and  IWs). 
In  the  split-plane  representation,  the  GPs  tend  to  be  separated  from  the 
other  experienced  subjects  (IWs  and  IPs)  on  the  vertical  dimension.  Data 
from  GPs  were  not  available  for  the  strafe  concepts,  but  in  that  represent¬ 
ation,  a  slight  separation  of  IPs  and  IWs  is  evident.  Therefore,  the  two- 
dimensional  MDS  solutions  of  individuals  provide  excellent  illustrations  of 
organization  of  individuals  along  an  "expertise"  dimension.  For  both 
representations,  this  is  the  horizontal  dimension  which  plots  less 
experienced  subjects  to  the  left  and  more  experienced  pilots  to  the  right. 
The  vertical  dimension  is  not  as  clearly  defined,  although  it  does  tend  to 
separate  the  different  groups  of  experts. 
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Figure  1.  Two-dimensional  MDS  solution  for  all  subjects  for  split-plane  concepts. 
I  -  Instructor  Pilots,  G  -  Air  National  Guard  Pilots,  U  -  Undergraduate 
Pilot  Trainees,  W  -  Instructor  Weapons  Systems  Officers. 
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In  an  attempt  to  further  delineate  the  different  groups  of  experts  in  an 
MDS  representation,  an  inter-subject  distance  matrix  was  derived  using  data 
from  only  the  expert  pilots.  The  two-dimensional  MDS  solutions  for  the 
expert  subset  are  presented  in  Figures  3  and  4.  In  the  split-plane 
representation,  IPs  and  IWs  are  nearly  linearly  separable  from  GPs.  This 
might  be  expected,  based  on  the  fact  that  IPs  and  IWs  are  involved  in 
teaching,  whereas  GPs  are  not.  Therefore,  this  dimension  could  be  labeled 
"teaching  experience,"  with  the  GPs  being  located  on  the  lower  end  of  the 

continuum  and  the  IPs  and  IWs  on  the  upper  end.  The  second  dimension  is  not 
as  clear  in  this  case.  In  the  strafe  representation,  the  horizontal 
dimension  tends  to  separate  the  IWs  from  IPs,  with  a  few  exceptions. 

Apparently,  the  IWs  and  IPs  are  not  as  easily  separated  as  other  groups. 

Table  17  shows  the  one-dimensional  MDS  solutions.  A  one-dimensional 
solution  should  extract  the  one  dimension  that  accounts  for  the  most 
variance.  In  Table  17,  subjects  are  presented  in  order  of  occurrence  in  the 
MDS  solution.  Values  assigned  to  each  pilot  are  transformed  coordinate 
values  and  are  an  indication  of  the  relative  distance  from  one  individual  to 
another.  In  both  split-plane  and  strafe  solutions,  the  largest  distance 

interval  occurs  between  the  group  of  UPs  and  experienced  subjects.  There  is 

one  exception  in  each  case,  however.  In  the  split-plane  solution,  UP-16  is 
in  the  expert  group  and  in  the  strafe  solution,  UP-12  is  in  the  expert 
group.  This  is  not  surprising  because  these  two  individuals  also  appear 
closer  to  the  expert  groups  in  the  two-dimensional  solutions.  Thus,  it 
seems  that  the  one-dimensional  solutions  ordered  individuals  along  the 
expertise  dimension. 

The  results  from  the  MDS  of  individual  subjects  clearly  help  to  define 
the  separate  groups  of  subjects  and  the  locations  of  individuals  in  relation 
to  the  groups.  The  previous  finding  that  the  more  experienced  pilots  agree 
more  with  each  other  and  have  well  defined  conceptual  structures  is 
supported  by  Figures  1  and  2,  in  which  expert  groups  tend  to  form  tighter 
clusters  than  do  UPs.  From  these  scaling  solutions,  two  dimensions  have 
been  labeled.  The  "expertise"  dimension  separates  UPs  from  all  other 
groups,  and  the  "teaching  experience"  dimension  separates  GPs  from  IWs  and 
IPs.  Also,  if  the  individuals  in  the  one-dimensional  solutions  are  ordered 
along  the  "expertise"  dimension,  then  this  ordering  may  prove  to  be  a 
possible  means  of  predicting  success  as  a  pilot  or  in  simply  determining  a 
student's  cognitive  "status"  in  relation  to  other  students  as  training 
progresses.  For  instance,  if  expertise  is  the  appropriate  label  for  the 
single  dimension  in  the  one-dimensional  solution,  individuals  that  are  close 
to  the  expert  end  of  the  continuum  should  have  organizations  of  flight- 
related  information  that  are  similar  to  those  of  experts.  In  general, 
representation  of  individual  pilots  in  multidimensional  space  provides  a 
concrete  illustration  of  relations  among  individuals  and  groups,  as  well  as 
an  indication  of  possible  dimensions  which  separate  these  individuals  and 
groups. 
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Figure  3.  Two-dimensional  MDS  solution  for  experienced  subjects  for  split-plane  concepts. 
I  -  Instructor  Pilots,  G  Air  National  Guard  Pilots,  W  -  Instructor  Weapons 
Systems  Officers. 


nstructor  Pilots,  W  -  Instructor  Weapons  System: 


Table  17  -  One-Dimensional  MD3  Solutions  for  Individuals 


Split-Plane  Maneuvers 


Low  Angle  Strafe 


Individual 


Position  on 
Dimension 


Individual 


Position  or. 
Dimension 


GP 

8 

0 

IP 

5 

0 

IW 

2 

3 

IW 

5 

26 

IP 

4 

29 

IP 

6 

51 

IP 

6 

36 

IP 

1 

52 

IP 

3 

38 

IP 

4 

60 

IX 

1 

40 

IP 

2 

62 

GP 

9 

42 

IW 

3 

88 

GP 

2 

43 

UP 

12 

91 

IP 

2 

46 

IP 

3 

94 

IW 

4 

48 

IW 

1 

96 

GP 

6 

50 

IW 

7 

98 

GP 

3 

59 

IW 

4 

111 

IP 

1 

61 

IW 

6 

119 

GP 

5 

62 

IW 

2 

123 

GP 

1 

66 

UP 

14 

202 

GP 

7 

67 

UP 

10 

218 

IP 

7 

69 

UP 

8 

230 

IP 

5 

75 

UP 

16 

246 

UP 

16 

91 

UP 

9 

249 

IW 

3 

92 

UP 

1 

256 

GP 

4 

103 

UP 

15 

260 

UP 

2 

184 

UP 

5 

267 

UP 

1 

198 

UP 

13 

273 

UP 

8 

198 

UP 

2 

274 

UP 

9 

227 

UP 

6 

278 

UP 

13 

235 

UP 

3 

286 

UP 

15 

236 

UP 

7 

300 

UP 

10 

241 

UP 

4 

310 

UP 

11 

247 

UP 

11 

326 

UP 

14 

251 

UP 

4 

256 

UP 

17 

268 

UP 

5 

271 

Note . 

UP 

6 

273 

IP-Instr uctor 

Pilots 

UP 

12 

278 

GP-Air  National  Guard 

Pilots 

UP 

3 

281 

UP-Undergraduate  Pilot 

Trainees 

UP 

7 

304 

IW-Instr uctor 

Weapons 

Systems  Officer 

EXPERIMENTAL  ANALYSES  OF  CONCEPTUAL  STRUCTURE 
Ordered  Recall:  A  Comparison  of  Networks  and  MDS 

The  MDS  techniques  and  the  recently  developed  scaling  technique  that 
produces  a  GWN  have  been  used  extensively  throughout  this  project  to 
investigate  the  organization  of  critical  flight-related  concepts  in  memory. 
Structural  representations  of  memory  have  been  generated  using  these 
techniques.  The  MDS  techniques  produce  spatial  representations  of  memory  in 
various  dimensions,  whereas  the  GWN  technique  produces  a  structural 
representation  consisting  of  nodes  and  links,  representing  concepts  and 
their  relationships,  respectively. 

As  previously  mentioned,  these  scaling  techniques  and  their  represent¬ 
ations  differ  in  several  ways.  The  MDS  techniques  involve  the  "fitting"  of 
a  set  of  psychological  distances,  such  as  those  derived  from  similarity 
ratings,  into  a  space  consisting  of  a  predetermined  number  of  dimensions. 
This  ,  rocess  of  manipulating  the  data  to  fit  the  dimensionality  results  in  a 
loss  of  some  information.  In  general,  the  information  lost  is  that 
concerning  specific  or  local  relationships  among  concepts.  On  the  other 
hand,  global  information,  such  as  clusters  of  several  concepts,  is 
augmented  through  this  technique.  Also,  from  a  MDS  spatial  representation, 
dimensions  in  which  concepts  are  organized  can  be  discovered  (e.g.,  temporal 
or  procedural  dimensions).  Another  advantage  of  the  MDS  technique  is  that 
from  the  solution,  a  distance  metric  can  be  derived.  These  distances  are 
useful  in  comparing  organizations  across  individuals  or  groups. 

The  GWN  provides  local  information  that  the  MDS  representation  lacks. 
Schvaneveldt  et  al.  (1932)  developed  the  GWN  algorithm,  and  Schvaneveldt 
and  Durso  (1931)  used  the  algorithm  to  scale  a  set  of  natural  concepts  such 
as  plants,  animals,  and  their  features.  To  illustrate  the  difference 
between  the  network  and  MDS  solutions,  they  positioned  the  nodes  in  the 
network  according  to  the  two-dimensional  MDS  solution.  The  concept, 
FEATHERS,  illustrates  the  fact  that  networks  are  better  able  to  represent 
concepts  that  are  connected  to  many  diverse  concepts.  In  the  MDS  solution  , 
FEATHERS  is  closely  related  to  the  concepts  CHICKEN,  FROG,  and  R03IN  in 
terms  of  distance  in  space,  while  it  is  distant  to  concepts  such  as  HAIR  and 
BIRD.  In  the  network,  however,  FEATHERS  is  linked  to  CHICKEN,  as  well  as 
HAIR  and  BIRD,  but  not  to  FROG  or  ROBIN.  Thus  the  GWN  captures  these 
additional  relationships  that  are  absent  in  the  MDS  solution.  Also,  in  the 
GWN,  atypical  category  members  are  not  connected,  whereas  they  may  be  close 
in  an  MDS  representation. 

In  general  then,  the  MDS  provides  global  information,  while  the  GWN 
provides  local  information.  An  important  issue,  however,  involves  the 
validation  of  one  of  these  structural  descriptions.  That  is,  which  scaling 
technique  produces  a  representation  that  is  most  similar  to  actual  memory 
structure?  To  compare  these  two  techniques,  an  ordered  recall  task  can  be 
used  in  which  the  order  of  the  list  is  derived  from  either  MDS  or  GWN 
representations. 
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Recall  order  has  been  used  frequently  to  generate  representations 
similar  to  those  produced  by  relatedness  ratings  (Adelson,  1981;  Chase  4 
Simon,  197  3;  McKeithen,  et  al.,  1981).  In  order  to  compare  memory  stuctures 
of  expert  and  novice  computer  programmers,  Adelson( 1 981 )  presented  lines  of 
Polymorphic.  Programming  Language  "(PPL)"  code  to  subjects  in  a  random 
fashion.  Subjects  were  asked  to  recall  the  lines  of  code.  Psychological 
distance,  in  this  case,  was  defined  as  the  number  of  intervening  items  in  a 
recalled  list.  Resulting  MDS  representations  showed  that  novice  programmers 
organize  according  to  syntax,  whereas  experts  organize  according  to 
procedure  or  function.  In  general,  it  is  assumed  in  these  studies  that  the 
order  in  which  a  list  of  randomly  presented  concepts  is  remembered 
corresponds  to  the  organization  of  those  concepts  in  memory. 

Chase  and  Simon  (  1  973  )  investigated  the  differences  between  chess 
players  of  different  skill  levels  in  a  memory  task  and  a  perception  task. 
In  the  memory  task,  subjects  had  to  reproduce  the  pieces  and  their  locations 
on  a  chessboard  after  viewing  the  setup  for  5  seconds.  They  found  that  for 
expert  chess  players,  recall  of  the  board  was  superior  when  the  pieces  were 
arranged  in  a  meaningful  fashion  than  when  the  pieces  were  arranged 
randomly.  Novices,  however,  did  not  benefit  from  a  meaningful  organization. 
This  finding  has  been  replicated  in  several  studies  (Engle  4  Bukstel, 
1978;  McKeithen,  et  al.,  1931),  suggesting  that  experts  benefit  from 
meaningful  organizations  as  opposed  to  random  arrangements. 

According  to  the  findings  of  these  studies,  a  meaningful  organization  is 
beneficial  to  recall  for  expert  subjects,  and  the  organization  that  corr¬ 
esponds  most  closely  to  actual  memory  should  produce  superior  recall.  Along 
this  line  of  reasoning,  a  list  of  words  or  concepts,  organized  in  a  meaning¬ 
ful  fashion  should  facilitate  recall  over  a  random  or  nonmeaningful  organ¬ 
ization,  when  the  subject  has  to  recall  the  words  in  the  order  in  which  they 
are  presented.  Also,  the  more  meaningful  the  organization  is  to  the  subject 
(the  more  similar  to  actual  memory  structure),  the  less  time  it  should  take 
for  perfect  ordered  recall. 

To  compare  the  MDS  and  GWN  representations,  recall  performance  can  be 
compared  u3ing  lists  of  concepts  generated  from  either  MDS  or  GWN  represent¬ 
ations.  The  MDS  list  would  consist  of  terms  ordered  in  such  a  way  that  each 
term  is  spatially  close  to  the  following  term  according  to  MDS,  but  is  not 
linked  to  the  following  term  in  the  network.  The  opposite  would  be  the  case 
for  the  GWN  lists.  Each  concept  would  be  linked  to  the  following  concept  in 
the  network,  but  the  terms  would  be  distant  according  to  MDS.  If  one  list 
is  recalled  in  significantly  fewer  trials  than  the  other,  then  it  is  likely 
that  the  representation  from  which  it  was  derived  should  correspond  more 
closely  to  the  actual  organization  of  those  concepts  in  memory. 

An  initial  study  was  done  U3ing  introductory  psychology  students  and 
natural  oncepts.  These  concepts  were  chosen  because  MDS  and  GWN  solutions 
were  already  available  for  these  concepts  (Schvaneveld t  4  Durso,  1981). 
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Method 


Subjects.  The  subjects  for  the  pilot  study  were  5**  undergraduates  from 
New  Mexico  State  University,  enrolled  in  an  introductory  psychology  class. 
The  subjects  volunteered  for  the  experiment  in  order  to  fulfill  course 
requirements. 

Apparatus.  The  stimulus  lists  were  presented  to  the  subjects  on  a  TERAK 
screen.  Subjects  also  typed  their  responses  using  the  TERAK  keyboard, 
positioned  below  the  screen. 

Materials.  Nine  stimulus  lists  (three  for  each  condition)  consisting  of 
12  words  each  were  used  in  this  experiment.  Concepts  were  identical 
to  those  used  by  Schvaneveldt  &  Durso  (1931),  and  the  GWN  and  MDS  represent¬ 
ations  derived  in  their  study  were  used  to  order  the  lists.  The  nine  lists 
are  reproduced  in  Table  18.  In  the  MDS-related  condition,  lists  were  derived 
by  locating  chains  of  concepts  that  were  close  to  each  other  in  the  MDS 
solution,  but  not  linked  in  the  network.  Network-related  lists  were 
generated  by  selecting  chains  of  concepts  that  were  linked  to  each  other  in 
the  network,  but  were  distant  in  the  MDS  solution.  Finally,  non-related 
lists  were  chains  of  concepts  that  were  neither  linked  to  each  other  nor 
closely  related  in  the  MDS  solution.  Tine  three  lists  within  each,  condition 
differed  in  terms  of  alternate  ways  of  generating  the  lists  (i.e.,  different 
starting  concepts  in  the  chain  or  a  choice  between  two  equally  distant 
concepts) . 

Procedure.  Subjects  were  randomly  assigned  to  one  of  three  conditions: 
(a)  GWN-related  list,  (b)  MDS-related  list  or  (c)  non-related  list.  Within 
conditions,  subjects  were  randomly  assigned  to  one  of  three  lists.  The 
purpose  of  using  three  different  lists  in  each  condition  was  to  determine  if 
id  iosyncracies  inherent  in  specific  lists  affects  time  to  recall  the  lists. 
The  subject  was  seated  in  front  of  a  TERAK  screen  and  presented  with  the 
instructions.  Then,  the  ordered  list  of  12  words  was  presented,  one  word  at 
a  time.  The  words  in  the  list  were  always  presented  in  the  same  fixed 
order.  Each  word  appeared  on  the  screen  for  1.5  seconds. 

After  all  words  were  presented,  the  subject  was  instructed  to  type  as 
many  words  as  could  be  recalled  in  the  same  order  as  they  had  been 
presented.  A  program  was  written  in  PASCAL  to  control  the  experiment. 
Spelling  and  typographical  errors  were  scored  as  correct  as  long  as  the 
words  were  sufficiently  close  to  the  target  words  to  be  recognized  by  the 
scoring  program.  Interresponse  times  were  also  recorded.  When  the  subject 
finished  the  list  or  could  recall  no  more,  the  words  were  checked  by  the 
computer.  If  all  12  words  were  not  recalled,  or  if  the  order  of  words 
was  incorrect,  another  trial  would  begin,  and  the  words  would  be  presented 
again.  This  procedure  continued  until  all  12  words  were  recalled  in  the 
correct  order.  The  total  number  of  trials  to  recall  the  list  was  recorded. 
The  maximum  possible  number  of  trials  was  12. 


Table  18  -  Ordered  Lists  Derived  for  Each  Condition 


Network-related  Lists  (linked  but  far  apart) 


LIST  1 

LIST  2 

LIST  3 

tree 

color 

frog 

robin 

hair 

living  thing 

bird 

feathers 

animal 

feathers 

bird 

dog 

hair 

robin 

hair 

color 

tree 

feathers 

green 

leaves 

bird 

frog 

green 

bat 

living  thing 

frog 

blood 

bat 

living  thing 

red 

mammal 

bat 

rose 

animal 

mammal 

flower 

MDS-related  Lists  (Close  but  not  linked) 


LIST  1 

LIST  2 

LIST  3 

hoof 

cottonwood 

feathers 

antler 

daisy 

frog 

hair 

plant 

robin 

deer 

leaves 

chicken 

mammal 

flower 

blood 

dog 

tree 

bird 

living  thing 

rose 

animal 

rose 

living  thing 

color 

tree 

mammal 

rose 

flower 

dog 

green 

leaves 

bird 

flower 

plant 

animal 

tree 

Non-related 

Lists  (not  linked  and 

far  apart) 

LIST  1 

LIST  2 

LIST  3 

cottonwood 

dog 

chicken 

chicken 

green 

leaves 

antler 

hoof 

hoof 

leaves 

feathers 

bird 

blood 

daisy 

green 

green 

bat 

mammal 

hair 

leaves 

plant 

frog 

mammal 

feathers 

daisy 

red 

deer 

feathers 

antler 

color 

deer 

robin 

dog 

red 

hair 

frog 
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Results  and  Discussion 


Mean  interresponse  time  was  calculated  for  each  subject  on  the  last 
trial.  The  mean  interresponse  times  for  each  condition  are  shown  in  Table 
19.  These  differences,  however,  are  not  significant.  On  the  other  hand, 
mean  number  of  trials  for  each  of  the  main  conditions  (See  Table  19)  do 
differ  significantly,  F(2,45)  =  10.019,  £<.001.  The  effect  of  different 
lists  within  conditions  on  nunber  of  trials  was  not  significant,  suggesting 
that  the  idiosyncr acies  within  a  particular  list  were  not  the  cause  of 
differences  found  between  conditions.  Linear  contrasts  done  on  mean  number 
of  trials  showed  that  the  differences  between  the  GWN  and  MDS  conditions 
(F(1,34)  =  17.772,  £<.001 )  and  between  the  GWN  and  non-related  conditions 
(F(1,j4)  =  24.322,  £<.001 )  were  significant.  The  differences  between  the 
MDS  and  non-related  conditions  were  not  significant. 


Table  19  -  Performance  Data  from  Ordered  Recall  Task 
Mean  Interresponse  Time  (sec) 


Network 

MDS 

Non-Related 

List  1 

1.76 

1.84 

1.65 

List  2 

2.91 

1.63 

2.04 

List  3 

3.21 

1.97 

2.87 

Mean 

2.63 

1.81 

2.19 

Mean  Number 

of  Trials  to 

Perfect 

Rec  al  1 

Network 

MDS 

Non-Related 

List  1 

3.50 

5.67 

6.50 

List  2 

4.17 

7.67 

7.00 

List  3 

3.67 

5.83 

9.16 

Mean 

3.78 

6.39 

7.55 

In  general,  the  results  of  this  study  suggest  that  GWN  better  captures 
the  aspects  of  memory  structure  responsible  for  aiding  ordered  recall. 
Subjects  in  the  GWN  condition  took  an  average  of  3.78  trials  to  recall  the 
lists  perfectly,  while  mean  number  of  trials  for  the  MDS  condition  (6.39) 
did  not  differ  significantly  from  the  mean  number  of  trials  for  non-related 
lists  (7.55). 


The  lists  organized  according  to  the  MDS  solution  were  no  easier  to 
recall  than  were  the  r.onmeaningful  lists.  This  finding  suggests  that  the 
MDS  representation  did  not  capture  the  aspects  of  memory  structure  that 
facilitate  ordered  recall.  The  superiority  of  the  GWN  list  can  be 
explained  based  on  the  properties  inherent  in  the  network.  In  a  task  such  a3 
this,  in  which  concepts  have  to  be  recalled  in  a  particular  order,  local 
relations  should  aid  recall  more  than  global  relations.  For  instance,  three 
concepts  found  in  the  GWN  lists,  TREE,  ROBIN,  and  BIRD  are  concepts  that  are 
linked  in  the  network,  but  are  distant  in  the  MDS  solution.  On  the  other 
hand.  HOOF,  ANTLER,  and  HAIR  are  three  concepts  from  the  MDS  lists.  These 
latter  concepts  are  globally  related  in  that  they  are  mammal  body  parts,  but 
are  not  linked  to  each  other  in  the  network.  Intuitively,  it  appears  that 
network-related  words  would  be  easier  to  remember  in  a  list  that  is  made  up 
of  a  series  of  chains  or  links  of  concepts  because  this  mimics  the  structure 
of  a  network. 

While  the  GWN  structure  was  superior  to  the  MDS  structure  in  this 
particular  task,  other  tasks  that  require  global  information  for  good 
performance  may  show  different  results.  Further  research  would  determine 
whether  MDS  structures  are  superior  in  other  situations. 
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Priming:  A  Comparison  of  Similarity  Ratings  and  MDS 


Previous  studies  have  indicated  that  visual  word  recognition  is 
performed  more  quickly  when  a  word  to  be  recognized  is  preceded  by  the 
presentation  of  an  associated  word.  Meyer,  Schvaneveldt,  and  Ruddy  (1975) 
performed  a  series  of  studies  that  showed  that  a  target  string  of  letters 
can  be  recognized  more  quickly  as  a  word  when  it  is  preceded  by  a  word 
that  is  associated  with  it.  For  instance,  the  string  of  letters  DOCTOR 
could  be  recognized  as  a  word  more  quickly  if  it  were  preceded  by  the  word 
NURSE  than  if  it  were  preceded  by  the  word  BREAD.  This  contextual  effect 
in  the  "lexical-decision"  task  (in  which  strings  of  letters  are  classified 
as  words  or  non-words)  was  described  in  terms  of  a  word-processing  model 
where  the  stimulus  letter  string  is  gr aphemically  encoded  and  transformed 
into  a  phonemic  representation  through  the  "grapheme-phoneme  correspondenc. e 
rules"  of  the  English  language.  Following  the  encoding  of  the  stimulus 
string,  lexical  memory  is  accessed  to  check  for  a  match  between  the  phonemic 
representation  and  some  item  stored  there.  Based  on  the  result  of  the 
retrieval  process,  the  stimulus  is  recognized  as  a  word  or  non-word  and  a 
yes/no  response  is  executed.  The  influence  of  the  semantic  context  was 
shown  to  affect  the  encoding  operation  as  opposed  to  the  retrieval  process. 
This  was  indicated  when  the  contextual  effect  was  shown  to  be  larger  for 
visually  degraded  letter  strings  than  for  letter  strings  of  unimpaired 
visual  quality. 

A  "spreading  activation"  theory  provides  an  explanation  of  the  semantic 
context  effect  displayed  in  a  word  recognition  task.  According  to  this 
theory,  individual  concepts  in  lexical  memory  are  represented  as  nodes  in  a 
network  structure.  The  activation  of  a  node  in  the  lexical  network  leads  to 
a  spread  of  activation  to  other  nodes  that  are  near  the  activated  node. 
The  organization  of  the  network  ensures  that  nearby  nodes  are  semantically 
(meaningfully)  related.  In  this  way,  spreading  activation  may  facilitate 
the  accessing  of  related  concepts  once  a  particular  node  has  been  activated. 

Studies  conducted  last  year  produced  a  list  of  concepts  relating  to 
flight  maneuvers.  These  concepts  were  used  to  construct  various 
representations  of  the  organization  of  the  concepts.  An  MDS  analysis  of 
similarity  ratings  for  pairs  of  the  concepts  resulted  in  a  spatial 
structure.  The  present  study  was  undertaken  to  evaluate  the  MDS 
representations  using  a  priming  procedure.  From  earlier  analyses  of  the  MDS 
solutions  (Schvaneveldt,  et  al.,  1932),  it  appears  that  MDS  solutions 
contain  more  information  than  do  the  ratings  from  which,  they  were 
constructed.  For  both  the  MDS  solutions  and  the  original  ratings,  some  pairs 
of  concepts  are  more  closely  related  than  are  other  pairs.  Since  priming 
presumably  reflects  underlying  memory  structure,  this  study  was  designed  to 
determine  whether  pairs  derived  from  MDS  solutions  would  lead  to  more  or 
less  priming  than  would  pairs  derived  from  the  original  ratings. 

The  effect  of  semantic  context  has  been  shown  to  generalize  across 
tasks,  as  subjects  were  consistently  faster  at  pronouncing  words  when  the 
words  were  preceded  by  associated  words  rather  than  unassociated  words  (see 
Meyer  et  al.,  1974).  In  this  study,  subjects  performed  a  similar 
pronunciation  task 


in  which  they  vocalized  target  words  that  followed  a  word  prime.  The  effect 
of  the  word  pair  relationship  (between  the  target  word  and  the  word  prime) 
on  reaction  time  was  examined.  The  relationship  between  words  in  a  pair 
varied  in  two  ways.  The  words  could  be  similar  or  dissimilar  based  on 
similarity  rating  averages,  or  the  words  could  be  near  to  or  far  from  each 
other  according  to  distances  derived  from  the  MDS  scaling  solution. 

Method 

There  are  435  word  pairs  in  all  possible  combinations  of  words  from  a 
list  of  30  3plit-plane  manuever  or  30  strafe  manuever  concepts.  The  terms 
from  each  pair  were  rated  for  similarity  on  a  scale  from  0  to  9  by  seven 
instructor  pilots.  From  the  averages  of  these  ratings,  an  MDS  solution  was 
obtained  that  produced  relative  distances  between  concepts,  representing  the 
strength  of  the  relationship  between  the  concepts  as  seen  by  the  pilots.  In 
this  way,  highly  related  words  were  given  shorter  MDS  distance  values,  while 
less  strongly  related  concepts  were  given  larger  MDS  distance  values. 

From  the  similarity  rating  averages  and  the  MDS  distances,  concept  pairs 
that  are  near  or  far  in  MDS  distance  and  high  or  low  in  similarity  were 
determined.  Pairs  of  concepts  that  had  a  high  similarity  rating  and  were 
near  to  each  other  in  the  network  solution  derived  from  MDS  were  selected  as 
"High-Near”  (H-N)  pairs.  Word  pairs  that  had  low  similarity  ratings  on  the 
average  and  were  most  distant  from  each  other  made  up  the  set  of  "Low-Far” 
(L-F)  pairs.  The  similarity  ratings  were  based  on  the  average  ratings  from 
the  eight  IPs.  The  MDS  distance  pairs  were  based  on  the  distances 
determined  from  a  MDS  solution  along  five  dimensions  for  the  averaged 
ratings. 

Concept  pairs  could  be  one  of  four  types:  (a)  H-N:  high  in  similarity 
and  near  in  MDS  distance,  (b)  L-F:  low  in  similarity  (dissimilar)  and  far  in 
MDS  distance,  (c)  H-F:  high  in  similarity  and  far  in  MDS  distance,  or  (d) 
L-N:  low  in  similarity  and  near  in  MDS  distance. 

Procedure .  Eight  instructor  pilots  served  as  subjects  (s)  and  were 
presented  with  two  terms,  one  at  a  time,  on  a  CRT  screen.  The  terms  were 
selected  from  a  list  of  30  split-plane  manuever  concepts  for  five  of  the 
subjects  and  30  low-angle  strafe  concepts  for  three  of  the  subjects.  The 
subjects'  task  was  to  read  aloud  the  second  concept  accurately  and  as 
quickly  as  possible.  The  vocal  response  time  was  measured  to  the  nearest 
millisecond  from  the  time  the  target  word  was  presented  until  the  subjects 
read  the  target  word  aloud.  The  subjects  were  presented  with  each  of  the  30 
concepts  as  targets  following  each  of  the  four  types  of  primes.  The  targets 
were  also  presented  following  the  prime,  NOTHING,  to  establish  a  neutral 
baseline  for  assessing  priming  effects. 
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A  single  trial  proceeded  as  follows.  A  warning  tone  alerted  the 
subjects  to  fix  their  gaze  on  the  CRT  screen  for  the  start  of  a  trial.  A 
word  prime  followed  the  offset  of  the  tone  and  was  displayed  for  0.5  second. 
The  screen  was  then  cleared,  and  there  was  a  1.0-second  delay  prior  to  the 
presentation  of  the  target  word.  The  target  word  remained  on  the  screen 
until  a  verbal  response  was  made.  The  subjects  spoke  into  a  microphone 
situated  7.5  to  10  centimeters  from  their  mouth.  A  delay  of  0.  1  second 
preceded  the*  warning  tone  for  the  next  trial. 

Each  subject  was  presented  with  150  concept  pairs  presented  in  a 
random  ordering.  The  150  pairs  consisted  of  all  30  concepts  paired 
with  the  five  different  prime  types.  The  subjects  completed  two  blocks  of 
trials,  with  each  block  having  a  different  random  ordering  of  the  150 
concept  pairs.  Response  times  were  recorded  by  the  computer  program  that 
controlled  the  experiment. 

Results 

The  mean  response  times  in  each  condition  are  shown  in  Table  20.  An 
analysis  of  variance  was  performed  on  the  pronunciation  response  times  and 
revealed  no  significance  for  the  priming  effect,  F(4,28)=0.86.  Apparently 
the  subjects  were  able  to  improve  their  performance  with  practice  as  they 
progressed  through  the  experiment.  This  practice  effect  is  indicated  by  the 
response  times  for  the  second  block  of  trials  being  significantly  lower  than 
the  response  times  for  the  first  block  of  trials,  F(1,7)=84.9,  £<.001. 


Table  20  -  Mean  Response  Times  (msec) 

for  Each 

Pr iming 

Condition. 

Priming  Condition 
(Similarity  -  MDS  Distance) 

Block  1 

Block  2 

Condition 

Mean 

High-Near 

461 

439 

450 

High-Far 

465 

438 

452 

Low-Near 

460 

444 

452 

Low -Far 

455 

442 

448 

Neutral 

469 

443 

456 

Block  Means 

462 

441 
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Di  scussion 


As  indicated  by  the  average  response  times  in  Table  20,  the  effect  of 

semantic  context  in  a  word  pronunciation  task  was  not  replicated  in  this 

study.  One  explanation  for  the  lack  of  facilitation  with  related  primes  may 
be  provided  by  the  spreading  activation  theory.  Each  subject  saw  each  of 
the  30  concepts  10  times  as  a  target  and  several  times  as  a  prime.  The 
resulting  spread  of  activation  from  activating  each  of  these  concepts  this 
frequently  may  be  so  large  that  all  of  these  concepts  are  highly  activated 
in  memory,  and  the  primes  on  individual  trials  do  not  lead  to  any  further 
activation  of  the  targets.  Some  support  for  this  interpretation  comes  from 
analysis  of  the  first  few  trials  in  the  experiment.  The  expected  priming 
effects  were  present,  but  with  so  few  trials,  there  were  no  significant 
effects. 

The  one  lesson  to  be  learned  from  this  study  appears  to  be  that  priming 
studies  cannot  use  procedures  that  require  extensive  repetition  of  stimulus 
materials.  With  a  limited  set  of  items  such  as  those  from  an  earlier  work, 

it  will  be  very  difficult  to  evaluate  memory  structure  using  priming 

methodology.  Other  tasks,  such  as  the  ordered  recall  task,  appear  to  be 
more  promising. 
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CRITICAL  CONCEPTS 


An  ongoing  concern  in  this  project  is  identifying  critical  concepts  that 
distinguish  expert  pilots  (instructors)  and  novice  pilots  (undergraduate 
trainees).  Earlier  work  on  this  problem  (  Sc  h  v  an  ev  e  Id  t  et  3l  .  1982) 

used  network  representations  to  compare  different  classes  of  pilots.  In 
network  representations,  concepts  are  depicted  by  nodes,  and  relations  are 
depicted  as  links  connecting  nodes.  The  links  are  assigned  values  which 
reflect  the  strength  cf  the  relationship  between  the  nodes.  Network 
representations,  as  mentioned  before,  are  extremely  useful  in  representing 
local  relationships  among  concepts.  The  networks  enabled  identification  of 
pairs  of  concepts  that  distinguished  experts  (instructor  and  guard  pilots) 
and  novices  (undergraduate  trainees).  Pairs  of  concepts  that  were  found  to 
discriminate  expert  3nd  novice  pilots  were  basically  of  two  types:  pairs 
between  which  there  existed  a  critical  association  for  experts  but  not  for 
novices  (the  pair  was  linked  in  the  network  representation  for  experts  but 
not  in  network  representation  for  novices)  and  pairs  between  which  there 
existed  a  critical  association  for  novices  but  not  for  experts. 

In  contrast  to  the  network  representations,  MDS  provides  a  more  global 
description  of  conceptual  structure.  The  problem  of  identifying  the 
critical  concepts  that  differentiate  novices  and  experts  can  thus  be  pursued 
on  a  more  global  level  through  representations  derived  from  multidimensional 
scaling.  With  MDS,  concepts  are  depicted  as  points  in  a  multidimensional 
space,  where  the  distance  between  concepts  along  a  dimension  represent 
psychological  distance  (i.e.,  two  concepts  that  are  close  along  a  given 
dimension  are  similar  along  that  dimension)  .  A  measure  of  distance  between 
each  concept  can  be  derived  from  the  concept  coordinates  in  multidimensional 
space.  Each  concept  can  then  be  represented  as  a  vector  of  distances  from 
the  concept  lo  every  other  concept  in  the  representation.  With  a  pool  of  30 
concepts  each  from  low  angle  strafe  and  split-plane  maneuvers,  each  concept 
can  be  represented  3s  a  vector  of  29  (n-1)  distances.  A  measure  of  the 
degree  to  which  UPs  and  IPs  agree  on  these  concepts  can  be  calculated  by 
correlating  each  concept  or  distance  vector  from  the  UPs  with  the  IPs. 
Tables  21  and  22  show  these  correlations  for  the  split-plane  maneuvers  and 
low  angle  strafe,  respectively.  As  can  be  seen  from  these  tables,  a  wide 
range  of  correlations  were  found.  For  example,  for  split-plane  maneuvers, 
high  agreement  between  IPs  and  UPs  was  found  for  the  concepts  AIRSPEED  and 
RELATIVE  ENERGY,  while  the  negative  correlations  found  for  ANGLE  OFF  and 
EXTENSION  reflect  extremely  low  agreement.  Similar  observations  can  be  made 
regarding  low  angle  strafe  concepts. 
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Table  21  -  Correlations  of  UPs  with  IPs 


Split-Plane  Maneuvers 

UP  Familiarity 
Rating 


Concept 

r_ 

2 

3 

Airspeed 

.81 

0 

0 

16 

Relative  Energy 

.81 

0 

4 

12 

Smash 

.76 

1 

1 

14 

Weapons  Parameters 

.71 

4 

12 

0 

Corner  Velocity 

.70 

1 

8 

7 

Power  Setting 

.70 

0 

0 

16 

Guns 

.70 

1 

14 

1 

Overtake 

.69 

0 

0 

16 

Acceleration 

.69 

0 

1 

15 

Switchology 

.64 

7 

5 

4 

Lift  Vector 

.62 

0 

4 

12 

g  Loading 

.60 

0 

1 

15 

Barrel  Roll 

.59 

0 

1 

15 

Verticle  Maneuver 

.53 

0 

2 

14 

Snapshot 

.52 

13 

3 

0 

Lag  Roll 

.47 

13 

3 

0 

3-9  Line 

.43 

12 

4 

0 

Aspect  Angle 

.43 

13 

3 

0 

6  O'clock 

.26 

0 

3 

13 

Quarter  Plane 

.25 

14 

2 

0 

Lag  Pursuit 

.22 

10 

4 

2 

Cutoff 

.20 

0 

0 

16 

Low  Yo  Yo 

.19 

10 

6 

0 

Pure  Pursuit 

.16 

9 

5 

2 

Lead  Pursuit 

.15 

10 

4 

2 

High  Yo  Yo 

.13 

8 

8 

0 

Radial  g 

.12 

12 

3 

1 

Heat 

.05 

13 

3 

0 

Angle  Off 

-.04 

4 

8 

4 

Ex  tension 

-.24 

13 

3 

0 

Note.  Familiarity  rating  entries  are  the  number  of  UPs  giving  each 
rating  to  each  concept. 

1- Totally  unfamiliar 

2- Familiar  but  not  used  in  flying 
"j-Used  in  flying 
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Table  22  -  Correlations  of  UPs  with  IPs 
Low  Angle  Strafe 

UP  Familiarity 
Rating 


Concept 

£ 

1_ 

2 

3 

Drift 

.76 

0 

6 

9 

Yaw 

.73 

0 

2 

13 

Fire 

.71 

1 

14 

0 

Burst 

.70 

3 

12 

0 

Pipper  Placement 

.69 

6 

9 

0 

Trim 

.67 

0 

1 

14 

Guns 

.65 

1 

14 

0 

Trigger 

.63 

2 

13 

0 

Final 

.61 

2 

4 

9 

Aim  Point 

.61 

1 

7 

7 

Stabilize 

.60 

2 

6 

7 

Tracking 

.57 

1 

10 

4 

Range 

.53 

1 

12 

2 

Bank 

.52 

0 

1 

14 

Bullet  Impact 

.50 

3 

12 

0 

Recovery 

.#9 

1 

a 

6 

Glide  Path 

.47 

0 

3 

12 

Run  In  Line 

.44 

14 

1 

0 

Pull  Up 

.44 

3 

5 

7 

Closure 

.42 

0 

5 

10 

Airspeed 

.39 

0 

2 

13 

Altitude 

.37 

0 

1 

14 

Pipper  Fixation 

.37 

7 

8 

0 

Dive  Angle 

.37 

1 

9 

5 

Ricochet 

.35 

6 

9 

0 

Aim  Off  Point 

.34 

12 

2 

1 

Foul  Line 

.26 

14 

1 

0 

Foul 

.21 

14 

1 

0 

Bunt 

.00 

14 

1 

0 

Walking 

-.07 

12 

3 

0 

Familiarity  rating  entries  are  the  number  of  UPs  giving  each 
rating  to  each  concept. 

1- Totally  Unfamiliar 

2- Familiar  but  not  used  in  flying 

3- Used  in  flying 


Concepts  that  show  low  correlations  are  probably  those  that  the  UPs  have 
not  learned.  At  any  rate,  these  concepts  are  not  organized  for  UPs  in  the 
same  way  as  for  the  instructors.  The  UPs  may  not  have  encountered  or  used 
these  concepts  in  training.  A  comparison  of  the  correlations  and  UP 
familiarity  ratings  supports  this  suggestion.  Generally,  those  concepts  that 
all,  or  most,  UPs  have  had  experience  with  during  training  correlate  highly, 
while  concepts  which  have  not  been  encountered  correlate  relatively  low. 
However,  there  are  exeptions  to  this  rule.  For  example,  all  UPs  have  used 
the  concept  CUTOFF  in  flying  (Table  21),  yet  they  correlate  relatively  low 
with  the  IPs.  Similarly,  all  UPs  have  had  either  flight  or  classroom 
experience  with  the  concept  6  O'CLOCK,  and  again,  they  correlate  relatively 
low  with  IPs.  For  low  angle  strafe,  similar  examples  include  DIVE  ANGLE, 
AIRSPEED,  and  ALTITUDE.  Although  UPs  have  encountered  these  concepts  in 
training,  their  organization  of  these  concepts  in  memory  still  greatly 
differs  from  that  of  the  IPs.  One  possible  reason  is  that  UPs  have  not 
learned  the  relationships  between  these  concepts  and  other  concepts.  During 
training,  they  may  have  encountered  these  concepts  in  specific  contexts  or 
situations  and  not  in  others. 

To  verify  further  the  critical  concepts  that  separate  novice  and  expert 
pilots,  pattern  recognition  techniques  (the  training  algorithm  described 
earlier)  were  applied  to  subsets  of  the  original  distances  derived  from 
multidimensional  scaling.  To  reiterate,  the  training  algorithm  operates  on 
the  distances  between  concepts.  Beginning  with  a  set  of  30  concepts,  each 
concept  has  29  distances  with  which  it  is  associated.  Distances  associated 
with  groups  of  concepts  were  systematically  removed,  starting  with  those 
concepts  that  were  most  highly  correlated.  By  removing  those  distances,  the 
remaining  subsets  contain  the  information  that  should  be  most  useful  in 
discriminating  between  IPs  and  UPs.  The  cutoffs  used  are  shown  as  the 
dotted  lines  in  Tables  21  and  22.  The  performance  of  the  training  algorithm 
on  subsets  of  the  original  distances  is  shown  in  Table  23.  As  can  be  seen 
from  Table  23.  training  algorithm  performance  did  not  greatly  deteriorate 
with  the  removal  of  all  concepts  that  correlated  at  or  above  .40.  This 
suggests  that  those  concepts  correlating  .40  and  above  did  not  contribute 
significantly  to  the  separation  of  the  IP  and  UP  groups,  and  the  concepts 
that  most  separate  the  UPs  and  IPs  are  those  that  correlate  below  .40.  One 
practical  consequence  of  these  findings  is  that  the  set  of  concepts  can  be 
considerably  reduced  without  seriously  affecting  the  discrimination  power  of 
the  task.  For  each  concept  eliminated,  data  collection  time  is  reduced  by 
the  number  of  remaining  concepts  because  of  the  testing  of  all  pairs  of 
concepts. 

Table  23  -  Pattern  Classification  Performance  for  Various  Cutoffs  (r) 
Split-Plane  Maneuvers  Low  Angle  Strafe 

Cutoff  (r) _ %  Correct  Cutoff  (r) _ ! j  Correct 


None 

.94 

None 

.94 

.70 

.92 

.60 

.96 

.60 

.91 

.50 

.99 

.50 

.96 

.40 

.90 

.40 

.89 

.30 

.48 

.20 

.84 
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CONCLUSIONS  AND  RECOMMENDATIONS 


Primary  Accomplishments 

This  project  has  produced  several  interesting  and  potentially  useful 
findings.  The  major  new  theoretical  effort  being  pursued  under  the  contract 
concerns  the  development  of  methods  for  obtaining  and  analyzing  networks  of 
concepts  from  empirical  data.  Networks  have  been  widely  postulated  as  the 
underlying  structure  of  concepts  in  semantic  memory.  Now  there  are  methods 
for  determining  the  nature  of  such  networks  without  relying  on  guesswork  and 
intuitive  judgments.  These  methods  should  prove  to  be  important  in  the 
continuing  analysis  of  structures  of  knowledge. 

In  the  domain  of  critical  flight-related  concepts,  it  has  been  shown 
that  systematic  methods  can  yield  valid  and  reliable  descriptions  of  the 
structure  of  these  concepts.  Further,  these  structures  can  be  used  to 
identify  individuals  as  members  of  groups  with  differing  training  and 
experience.  Of  particular  interest  are  the  findings  showing  more  accurate 
classification  of  individuals  into  groups  using  structural  representations 
of  concepts  than  using  the  original  data  from  which  the  structures  are 
derived.  Multidimensional  spatial  structures  lead  to  the  most  accurate 
classification,  but  network  structures  also  lead  to  more  accurate  classifi¬ 
cation  than  Jo  the  direct  judgments  obtained  from  various  groups  of  flying 
personnel.  Since  the  network  representations  are  based  only  on  information 
about  which  concepts  are  linked  to  which  other  concepts,  the  classification 
performance  with  network  structures  is  particularly  noteworthy.  Perhaps  as 
the  testing  of  a  metric  for  the  network  structure  proceeds,  classification 
with  networks  will  approach  the  level  of  performance  found  using  the  MDS 
structures. 

Further  specific  areas  of  disagreement  have  also  been  identified  in  the 
structures  possessed  by  expert  and  novice  pilots.  These  specific  differ¬ 
ences  may  deserve  special  attention  in  lead-in  fighter  training.  The 

structures  themselves  may  also  prove  to  be  useful  in  the  academic  program 

since  they  provide  some  graphic  examples  of  the  differences  in  the  ways 

novices  and  experts  think  about  critical  flight  concepts. 

One  major  urea  investigated  in  the  past  year  concerns  the  experimental 
verification  of  the  structures  that  were  identified.  An  ordered  recall  task 
was  developed  to  compare  MDS  and  network  structures.  In  the  preliminary 
test,  the  network  representation  facilitated  recall  while  the  MDS  represent¬ 
ation  did  not.  Although  this  finding  is  suggestive,  it  should  not  be  taken 
as  definitive  for  two  reasons.  First,  the  ordered  recall  task  itself  may  be 
biased  in  favor  of  networks  because  the  task  requires  recalling  items  in  a 
designated  order.  Second,  further  work  with  the  task  using  fighter  pilots 
is  underway,  and  that  test  may  show  different  results.  At  the  very  least, 
however,  the  initial  experiment  did  provide  some  validation  of  the  network 
structure  in  a  memory  performance  test.  This  validation  serves  to  reinforce 
earlier  findings  that  the  structures  being  discovered  do  reflect  some 

important  aspects  of  the  underlying  organization  of  flight-related  concepts. 


A  final  major  accomplishment  this  year  has  been  the  analysis  of  larger 
units  of  conceptual  structure  in  the  form  of  scripts.  This  work  is  reported 
in  detail  in  a  separate  technical  paper  (Maxwell  4  Schvaneveldt ,  in  press). 
In  brief,  this  work  has  shown  that  pilots  do  have  script-like  organizations 
for  the  particular  actions  that  go  together  to  make  up  various  maneuvers. 
The  experiments  showed  that  expert  pilots  (fighter  lead-in  IPs)  are  able  to 
integrate  actions  that  constitute  parts  of  scripts,  and  they  also  show  clear 
evidence  of  elaborating  presented  actions  with  other  actions  that  belong  to 
the  script  but  were  not  presented.  Novice  pilots  (UPs)  show  quite  different 
effects.  They  are  not  able  to  integrate  actions  that  belong  together  as 
completely  as  do  the  experts,  and  they  show  no  evidence  of  elaboration  of 
scripted  activities.  Naive  subjects  (undergraduate  psychology  students) 
show  still  another  pattern  of  results.  As  would  be  expected,  they  show  no 
integration  and  no  elaboration.  Of  course,  they  should  have  no  knowledge  of 
the  scripts  and,  therefore,  should  not  show  script  effects. 

Directions  for  Future  Work 

The  next  reporting  period  will  be  devoted  to  additional  work  on  the 
scaling  procedures  used  and  developed  throughout  the  contract  period. 
Further  experimental  studies  are  planned  to  evaluate  the  results  of  the 
scaling  procedures. 

One  major  new  analysis  is  planned  for  the  next  year.  The  new  analysis 
will  be  concerned  with  a  procedural  analysis  of  decision  making  in  fighter 
pilots.  The  intent  is  to  develop  a  production  system  model  to  simulate  the 
decisions  made  by  fighter  pilots  in  particular  combat  situations  and/or  in 
executing  particular  maneuvers.  The  rationale  behind  this  project  is  that  a 
successful  simulation  of  pilot  decisions  will  constitute  a  model  of  the 
decision-making  process  as  well  as  the  knowledge  necessary  to  make  such 
decisions. 

Recommendations 

The  work  accomplished  under  this  contract  has  provided  a  detailed  anal¬ 
ysis  of  the  conceptual  structure  of  critical  flight  information  in  fighter 
pilots  as  well  as  correlational  and  experimental  verification  of  the  valid¬ 
ity  of  the  structures.  These  structures  may  be  useful  in  training  programs 
for  students  attempting  to  acquire  these  conceptual  structures.  Also,  the 
representations  themselves  may  prove  to  be  useful  as  training  aids.  The 
network  analysis,  for  example,  shows  how  expert  fighter  pilots  organize  the 
concepts  involved  in  particular  maneuvers.  To  the  extent  that  the  network 
representation  provides  an  understandable  representation  of  experts' 
knowledge  structures,  students  may  find  it  useful  in  learning  about  the 
maneuvers. 
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From  a  somewhat  different  angle,  specific  differences  have  been 
identified  in  the  conceptual  structures  of  students  and  expert  fighter 
pilots.  In  particular,  the  differences  show,  in  part,  what  experts  know 
that  students  do  not  and  what  misconceptions  the  students  may  have  acquired 
from  earlier  training  or  from  other  life  experiences.  These  areas  of 
difference  should  receive  special  attention  in  the  training  program  for 
fighter  pilots. 

Finally,  work  in  classifying  individuals  based  on  their  conceptual 
structure  suggests  further  work  in  attempting  to  predict  the  success  of 
future  fighter  pilots  based  on  the  conceptual  structures  students 
demonstrate  early  in  training.  It  may  be  necessary  to  study  the  structures 
associated  with  a  different  set  of  concepts  than  those  used  in  the  present 
investigation.  For  example,  perhaps  some  concepts  relating  to  attitude  and 
motivation  should  be  included  along  with  concepts  relating  to  the  operation 
of  aircraft.  The  classificatioi  techniques  developed  appear  to  be  very 
sensitive  to  differences  in  cognitive  structures,  and  they  may  well  provide 
some  predictive  power  for  organizing  the  training  program  to  produce  maximum 
benefit  for  those  who  are  likely  to  benefit  the  most  from  fighter  training. 

In  a  more  general  vein,  attempts  to  define  the  cognitive  structures 
involved  in  successful  operation  of  fighter  aircraft  should  serve  to 
complement  other  research  concerned  with  perceptual  processes  and  motor 
skills.  The  investigation  of  knowledge  organization  in  pilots  seems  to  have 
been  relatively  neglected  in  attempts  to  apply  psychology  to  the  under¬ 
standing  of  the  pilot  and  the  pilot’s  task.  Recently,  psychologists  and 
workers  in  the  field  of  artificial  intelligence  have  made  some  important 
strides  toward  a  more  complete  understanding  of  knowledge  representation  and 
the  process  involved  in  the  use  of  knowledge.  It  is  believed  that  this  work 
contributes  to  the  application  of  these  recent  developments  to  the  task  of 
understanding  the  knowledge  of  fighter  pilots.  If  cognitive  3kills  are 
found  to  be  at  least  as  important  as  perceptual-motor  skills  for  successful 
pilots,  it  is  imperative  to  continue  to  advance  the  understanding  of  the 
nature  of  these  cognitive  skills. 
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